Tianshou is a deep reinforcement learning library built on PyTorch that provides both low-level interfaces for algorithm research and high-level interfaces for practical application.
The library addresses the need for a flexible, performant framework that unifies diverse RL paradigms. It separates learning algorithms from policies through distinct abstractions, establishing clear type-level distinctions between on-policy, off-policy, and offline algorithms. This modular design allows researchers to implement custom algorithms while enabling practitioners to train existing algorithms on custom environments without deep framework knowledge.
Tianshou suits teams building production RL systems or conducting algorithm research where code clarity and extensibility matter. The library's scope spans online reinforcement learning, offline RL, experimental multi-agent RL support, and model-based approaches, making it appropriate for projects that may evolve across these domains. Practitioners should note that the library underwent a major redesign that breaks backward compatibility with earlier versions, requiring migration effort for existing codebases but establishing clearer, more consistent interfaces going forward.
The project maintains active development with regular refinements to its core abstractions and documentation. The codebase shows sustained attention to API design, with parameter naming and class hierarchies revised for intuitiveness. The library receives ongoing updates to support contemporary RL research directions, including experimental features for emerging areas like multi-agent systems and model-based methods.
