AI Infra Book is an open-source technical book that teaches quantitative analysis and system design for large language model infrastructure, grounded in hardware constraints and model architecture.
The book addresses the gap between software engineers who understand LLM applications but lack insight into the infrastructure that runs them. It teaches that infrastructure decisions directly impact latency, cost, and feasibility of LLM-based products. The approach centers on deriving system design from first principles: starting with task requirements and quality constraints, then quantifying compute, storage, communication, and dependency needs against hardware capacity, bandwidth, and compute capability. The book emphasizes that estimation errors are common—forgetting KV cache in calculations, confusing peak compute with bandwidth-limited performance, or overlooking inter-GPU communication—and that conclusions can diverge by orders of magnitude from reality. The methodology repeatedly asks five questions: what data moves, how much, how many times, through which path, and who must wait for it.
The book suits software engineers building LLM applications who want to understand the infrastructure layer, as well as those working on LLM serving systems, training infrastructure, or hardware acceleration. It covers LLM inference and training system design, distributed systems, GPU and accelerator optimization, KV-cache management, mixture-of-experts architectures, and datacenter networking. The book is presented as a companion to an earlier work on AI agents, extending that foundation into infrastructure concerns. The material is available as Markdown source files in the repository and as a PDF built with XeLaTeX, with the PDF recommended for reading due to extensive formulas, tables, and cross-references that render incompletely in GitHub's Markdown display.
Development activity shows consistent engagement with the material. The repository accepts corrections and improvements from readers through issue submission. The book receives regular updates published to releases, with the PDF automatically rebuilt after changes to the main branch. The project maintains a homepage separate from GitHub for broader accessibility.
