Reasoning from Scratch is an educational codebase that teaches how to implement a reasoning large language model in PyTorch from the ground up.
The repository addresses the challenge of understanding how reasoning capabilities are added to language models, moving beyond theoretical descriptions to hands-on implementation. It starts with a pre-trained base LLM and progressively adds reasoning functionality through code, allowing learners to see exactly how techniques like chain-of-thought reasoning, test-time compute scaling, and reinforcement learning from human feedback work in practice. The approach mirrors methods used in production reasoning models such as DeepSeek R1 and GPT-5 Thinking, while remaining tractable for educational purposes.
This project suits developers and researchers who want to understand reasoning LLMs deeply rather than just use them as black boxes. It is designed for those willing to engage with step-by-step implementations in Jupyter notebooks and PyTorch code. The repository is particularly valuable for anyone building or fine-tuning their own reasoning models, or for educators teaching how modern reasoning systems function. It also includes code for loading weights from existing pretrained models, bridging theory and practical application.
The project is structured as the official code repository accompanying a published book, with a troubleshooting guide provided for users. The codebase is organized progressively through chapters, with Chapter 2 covering environment setup and Python package management to help users get started.
