xlite-dev/LeetCUDA

LeetCUDA is a CUDA learning resource that teaches GPU programming fundamentals through practical kernel implementations and PyTorch integration.

The project addresses the challenge of learning CUDA by providing a hands-on collection of over two hundred kernel implementations covering core GPU concepts. It progresses from foundational CUDA programming patterns to advanced topics including tensor core operations, half-precision general matrix multiplication (HGEMM), and Flash Attention 2 matrix multiply accumulate (MMA) operations. The approach combines educational notes with runnable code examples that demonstrate how these kernels work in practice, bridging the gap between CUDA theory and real-world GPU computation patterns used in modern deep learning frameworks.

Developers new to GPU programming will find this resource most valuable, particularly those wanting to understand how PyTorch operations map to underlying CUDA implementations. The collection of kernel examples serves as both a reference and a learning tool for anyone building GPU-accelerated applications or seeking to optimize existing code. The focus on modern CUDA versions and contemporary techniques like Flash Attention makes it relevant for developers working with current deep learning infrastructure rather than legacy GPU code.

The project maintains active development with regular additions to its kernel collection and documentation. Commits show consistent expansion of the codebase with new kernel implementations and refinements to existing examples. The repository demonstrates ongoing engagement with the learning material through iterative improvements and supplementary notes that clarify complex GPU programming concepts.