OpenAI Baselines is a Python-based repository containing high-quality implementations of reinforcement learning algorithms designed to serve as reference implementations for the research community. The project is currently in maintenance mode, with the README explicitly stating that users should expect bug fixes and minor updates rather than active feature development. The repository was created to make it easier for researchers to replicate published results, refine existing algorithms, and build new ideas on top of established baselines.
The repository implements multiple state-of-the-art reinforcement learning algorithms including A2C, ACER, ACKTR, DDPG, DQN and its variants, GAIL, HER, PPO1, PPO2, and TRPO with MPI support. The DQN implementation and its variants are documented as achieving scores roughly on par with published papers, establishing the repository as a reliable benchmark for comparing new approaches against existing ones. The codebase is written in Python and requires Python 3.5 or higher, along with system dependencies including CMake, OpenMPI, and zlib.
The repository supports training on multiple types of environments. It includes extensive support for MuJoCo physics simulation environments, though MuJoCo itself is proprietary and requires a separate license. The baselines also work with Atari environments, with the DQN on Atari serving as a classic benchmark. Training is facilitated through command-line interfaces that allow users to control hyperparameters for both network architecture and learning algorithms. For example, users can train PPO2 on MuJoCo Humanoid or DQN on Atari Pong with customizable network configurations and learning parameters.
The repository provides functionality for saving, loading, and visualizing trained models, though the README notes that the serialization API is not fully unified across all algorithms. Models can be saved and restored using command-line options, and training metrics are logged to temporary directories by default, with the ability to customize logging paths through command-line arguments or environment variables. The repository includes visualization tools for learning curves and other training metrics.
According to GitGenius activity tracking, the repository has a median issue and pull request response latency of approximately 38,061 hours with a mean of 40,979 hours, indicating that while the project is maintained, response times are extended. The most active tracked contributors include KeepOnCarvingHuang, PostResPerditas, and VincentOracle, each with one recorded event. The repository shares overlapping contributors with major projects including Microsoft's VSCode and TypeScript repositories as well as the Rust language repository, suggesting cross-pollination with other significant open-source initiatives.
The repository includes comprehensive benchmarks for both MuJoCo environments at 1 million timesteps and Atari environments at 10 million timesteps, with results available on the project's GitHub page. The codebase supports TensorFlow versions 1.4 through 1.14 on the master branch, with a separate tf2 branch providing TensorFlow 2.0 support. Installation requires cloning the repository and installing the baselines package along with the appropriate TensorFlow version.
