LLMs from Scratch CN is an educational resource that teaches how to build large language models from the ground up using only Python fundamentals.
The resource addresses the challenge of understanding how large language models work by providing step-by-step construction guides for specific models. Rather than treating these systems as black boxes, it takes a hands-on approach where learners build working implementations of GLM4, Llama3, and RWKV6 from scratch. This practical methodology helps developers grasp the underlying principles and architecture decisions that define modern language models.
This resource suits developers and researchers who want to move beyond using pre-built models to actually understanding their internals. It works well for those with basic Python knowledge who are willing to engage with detailed implementation walkthroughs. The focus on building multiple distinct architectures means learners encounter different design patterns and can understand both common principles and model-specific innovations. This is particularly valuable for anyone planning to fine-tune, optimize, or extend language models in their own work.
The project maintains active development with regular updates to its educational materials and implementations. The codebase uses Jupyter Notebooks as its primary format, making it accessible for interactive learning and experimentation. The repository demonstrates consistent engagement with its subject matter through ongoing refinement of its tutorials and code examples.
