Miles is a reinforcement learning framework for large-scale language and vision model post-training that pairs high-throughput rollout generation with scalable distributed training.
The framework addresses the challenge of efficiently training large language and vision models using reinforcement learning at enterprise scale. It integrates SGLang for fast rollout generation and Megatron-LM for distributed training, combining these components to handle the computational demands of trillion-parameter model post-training. The tool provides precision, stability, and observability features designed specifically for reinforcement learning workflows at this scale. A PyTorch FSDP2 backend is also available for teams preferring to train HuggingFace model implementations directly, though the largest models and most optimized recipes use the Megatron-LM backend.
Teams should adopt this tool if they are running production reinforcement learning workloads on large language or vision models and need infrastructure that scales to enterprise requirements. It suits organizations performing post-training optimization where throughput and training stability are critical. The framework supports a range of models and includes specialized features like on-policy distillation and token-in-token-out training patterns. The project maintains day-zero support for frontier models across different hardware platforms, including both NVIDIA and AMD accelerators.
Development activity shows consistent feature expansion with regular releases introducing new capabilities for distributed training and model support. The project maintains active documentation and a dedicated community channel. The team regularly publishes technical blog posts detailing new features and performance optimizations. The framework receives updates addressing both hardware-specific optimizations and algorithmic improvements for reinforcement learning at scale.
