Parameter Golf is a machine learning competition that challenges participants to train the smallest language model possible within a 16MB size constraint.
The challenge optimizes for model quality under strict parameter limits. Participants train language models evaluated by compression performance on the FineWeb validation set, measured in bits per byte. The constraint forces exploration of unconventional approaches: test-time compute, aggressive parameter tying, depth recurrence, low-rank training, novel compression schemes like low precision and quantization-aware training, and creative techniques such as test-time training or long context handling. This framing differs from related speedrunning challenges that optimize for training time or dataset efficiency; here the objective is extracting maximum performance from a fixed parameter budget regardless of data or compute resources.
The competition suits researchers and engineers interested in model compression, efficient architectures, and pushing the boundaries of what neural networks can achieve under resource constraints. It appeals to those who enjoy optimization puzzles and want to explore unconventional model designs. The challenge provides compute credits to reduce barriers to entry, making it accessible beyond those with personal GPU resources. Participants range from students to experienced researchers, with OpenAI explicitly targeting early-career talent through the competition format.
The project maintains an active leaderboard accepting submissions that meet the official constraints of 10 minutes on 8xH100s, while also welcoming non-record submissions that explore parameter-limited performance without compute restrictions. The challenge runs for a defined period with a structured participant form for attribution and outreach. OpenAI frames the competition as a talent identification mechanism aligned with their hiring and research priorities, positioning exceptional performance as a pathway to professional opportunities.
