ThunderKittens is a framework for writing fast deep learning kernels in CUDA.
The framework addresses the challenge of developing high-performance GPU kernels by providing tile-based primitives that align with how modern GPU hardware actually operates. Rather than treating GPUs as monolithic matrix processors, ThunderKittens recognizes that individual cores efficiently handle small tile operations around 16x16 values. The framework abstracts away low-level complexity while remaining natively embedded in CUDA, allowing developers to write performant kernels without sacrificing the ability to drop down to raw CUDA when needed.
ThunderKittens suits teams building production deep learning systems who need custom kernels for attention mechanisms, matrix operations, or other compute-intensive layers. It is designed specifically for NVIDIA GPUs; developers targeting AMD hardware should consider the AMD variant instead. The framework prioritizes three qualities: simplicity in kernel authoring, extensibility through native CUDA integration, and speed comparable to hand-optimized kernels. The project demonstrates this capability through implementations like FlashAttention-3, which shows that kernels written with the framework can match or exceed performance of kernels written from scratch.
The project maintains active development with recent releases bringing support for newer GPU architectures including Blackwell and Vera Rubin, along with emerging precision formats like MXFP8 and NVFP4. The repository structure underwent significant reorganization to move away from Python package distribution toward individual kernel compilation with colocated Makefiles and tests. Support for older GPU generations has been explicitly narrowed, with the maintainers no longer actively supporting Ampere architecture. The codebase is used in production by multiple AI companies for both training and inference workloads.
