llm-from-scratch is an educational workshop that teaches you to build a GPT language model from scratch by writing every component yourself in PyTorch.
The project addresses the challenge of understanding how language models actually work by stripping away complexity and focusing on essentials. Rather than studying a full-scale implementation, you build a roughly 10-million-parameter model that trains on a laptop in under an hour. The approach guides you through writing a character-level tokenizer, the complete transformer architecture including embeddings and self-attention, a training loop with loss computation and optimization, and text generation with sampling. By the end you have working tokenizer, model, training, and generation code that you wrote yourself, trained on Shakespeare text.
This suits anyone wanting hands-on understanding of transformer internals without needing machine learning expertise beforehand—just Python comfort and a laptop. The workshop is structured to complete in a single session, making it accessible for learners who want to move beyond high-level explanations. It works on Mac, Linux, or Windows, with automatic GPU support for Apple Silicon or NVIDIA hardware, and also runs in Google Colab. The project explicitly positions itself as a stripped-down alternative to nanoGPT, which targets reproducing GPT-2 at 124 million parameters; this version scales down to essentials and trains faster on consumer hardware.
Development activity shows consistent engagement with documentation and examples. The project maintains organized, sequential learning materials across six parts with clear explanations of concepts at each stage. Contributions focus on keeping the codebase minimal and the learning path coherent rather than adding features.
