PettingZoo is a Python library for multi-agent reinforcement learning that provides a standard API for building and testing multi-agent environments.
The library addresses the fragmentation in multi-agent RL research by offering a unified interface modeled on the Agent Environment Cycle framework. This approach allows a single API to cleanly support diverse multi-agent scenarios—cooperative, competitive, and mixed-sum games—while minimizing common implementation bugs. The tool includes reference environments across multiple families: Atari multi-player games, cooperative graphical games called Butterfly, classical games including card and board games, and SISL environments for cooperative tasks.
Teams should adopt PettingZoo if they are conducting multi-agent RL research and want to avoid building custom environment infrastructure. The library suits projects ranging from algorithm development to curriculum learning and self-play training. The tool is designed as a multi-agent counterpart to Gymnasium, making it a natural choice for researchers already familiar with that ecosystem. Installation is flexible, allowing users to install only the dependencies needed for specific environment families rather than all at once.
Development activity shows consistent engagement with the research community. The project maintains active coordination through a public Discord server used for development planning. Documentation is comprehensive, including tutorials for environment creation, custom environment examples, and integration guides with popular training frameworks like CleanRL, Tianshou, and AgileRL. The maintainers officially support Linux and macOS while accepting Windows-related contributions without formal support commitments.
