Colibri is an inference engine that runs frontier mixture-of-experts models with hundreds of billions to trillions of parameters on consumer and heterogeneous hardware using pure C with zero engine dependencies.
The tool solves the problem of running extremely large models on limited hardware by treating storage, RAM, and VRAM as a single inference hierarchy. Rather than requiring all model weights to fit in fast memory, it streams experts from disk on demand, allowing models far larger than any single machine's capacity to run on ordinary consumer GPUs. The engine implements AI memory multitiering across the entire software and hardware boundary, optimizing model formats, storage I/O, placement, scheduling, and CPU-GPU overlap to reduce inference costs and hardware requirements.
Colibri suits researchers and practitioners who own consumer or heterogeneous hardware and want to run state-of-the-art large models without cloud infrastructure. It works with eight model families spanning from 7B to 2.8T parameters, each implemented as a single C file with a shared command-line interface for chat, serving, and web dashboards. The project prioritizes semantic correctness over speed—it makes no silent changes to model precision or router behavior when memory is insufficient, only reducing throughput. This makes it suitable for applications where model fidelity matters more than latency guarantees.
Development shows sustained focus on systems-level optimization across the full inference stack. The project maintains a web dashboard with live metrics and expert visualization, allowing developers to observe routing patterns and memory tier usage in real time. Work spans multiple model families and hardware configurations, indicating active testing against diverse setups. The codebase remains deliberately experimental, treating the engine as an open research platform where aggressive systems ideas must earn inclusion through reproducible end-to-end measurements rather than theoretical promise.
