The DeepSeek-R1 repository contains the implementation and models for DeepSeek's first-generation reasoning models, specifically DeepSeek-R1-Zero and DeepSeek-R1, along with six distilled variants. These models represent a significant advancement in large language model reasoning capabilities, trained using large-scale reinforcement learning applied directly to base models without requiring supervised fine-tuning as a preliminary step. DeepSeek-R1-Zero demonstrates that reasoning capabilities can emerge purely through RL optimization, marking what the repository describes as the first open research validating this approach. The full DeepSeek-R1 model incorporates cold-start data before RL training and achieves performance comparable to OpenAI-o1 across mathematics, code, and reasoning tasks.
The repository's technical approach involves a multi-stage pipeline combining two RL stages designed to discover improved reasoning patterns and align with human preferences, followed by two SFT stages that seed the model's reasoning and non-reasoning capabilities. DeepSeek-R1-Zero and DeepSeek-R1 are both built on the DeepSeek-V3-Base architecture, featuring 671 billion total parameters with 37 billion activated parameters and a 128K token context length. The models naturally develop reasoning behaviors including self-verification, reflection, and generation of extended chain-of-thought reasoning, though DeepSeek-R1-Zero encounters challenges such as endless repetition, poor readability, and language mixing that the full DeepSeek-R1 addresses.
A key contribution is the distillation of reasoning patterns into smaller models, with six open-sourced distilled checkpoints ranging from 1.5B to 70B parameters based on Qwen2.5 and Llama3 series architectures. DeepSeek-R1-Distill-Qwen-32B notably outperforms OpenAI-o1-mini across various benchmarks, achieving state-of-the-art results for dense models. The repository demonstrates that smaller models can effectively learn reasoning patterns from larger models, enabling practical deployment of advanced reasoning capabilities.
Evaluation results show DeepSeek-R1 achieving strong performance across multiple benchmark categories. On English benchmarks, it scores 90.8 on MMLU, 84.0 on MMLU-Pro, and 92.2 on DROP. For code tasks, it achieves 65.9 pass@1 on LiveCodeBench and 96.3 percentile on Codeforces. On mathematics benchmarks, DeepSeek-R1 reaches 79.8 on AIME 2024 and 97.3 on MATH-500. Chinese language performance includes 92.8 on CLUEWSC and 91.8 on C-Eval.
The repository is actively maintained with significant community engagement. The codebase shares contributors with major projects including Microsoft's VSCode and TypeScript repositories, as well as the Rust language project, indicating cross-pollination with significant open-source ecosystems. The repository is released under the MIT license and provides access to models through Hugging Face, with a detailed technical paper available for reference.
