TurboFieldfare is a Swift and Metal runtime that enables inference of Google's Gemma 4 26B-A4B language model on Apple Silicon Macs with as little as 8 GB of RAM, using approximately 2 GB of memory during operation. The project addresses the challenge of running large language models on resource-constrained hardware by implementing a custom streaming architecture that keeps only essential model components in memory while fetching expert weights on-demand from storage.
The core innovation behind TurboFieldfare is its expert streaming approach. Rather than loading the entire 14.3 GB model into memory, the runtime maintains the 1.35 GB shared core and FP16 key-value cache in RAM, then streams only the specific mixture-of-experts weights needed for each token generation from the SSD. This architectural choice allows the 26-billion-parameter model to run on entry-level Apple Silicon machines. The model uses MLX affine 4-bit quantization with group 64 for weights, 8-bit router weights, and 4-bit shared and routed experts, with approximately 3.88 billion parameters active per token.
The project is written entirely in Swift and Metal, making it model-specific rather than a wrapper around existing frameworks like MLX or llama.cpp. TurboFieldfare provides six distinct products: the core TurboFieldfare library containing the runtime and Metal kernels, a native macOS application for installation and text generation, a decode service executable that manages the model and Metal resources, a command-line interface for chat and completion tasks, an experimental OpenAI-compatible loopback server, and a streaming model installer with verification capabilities. The native Mac app handles Gemma's chat formatting automatically, allowing users to enter instructions without manual prompt engineering.
Performance varies significantly by hardware. On an 8 GB M2 MacBook Air, measured decode speeds range from 5.1 to 6.3 tokens per second, while a 24 GB M5 Pro achieves 31 to 35 tokens per second. These measurements represent reference points rather than performance ceilings, as throughput depends on prompt length, generation length, page-cache state, and hardware configuration. The project maintains detailed community benchmark results and provides a guide for contributors to add their own measurements.
The inference engine operates through a sophisticated layer-by-layer process. At each transformer layer, Metal computes attention and router operations from resident weights. The CPU then uses the router's top-8 expert IDs to plan against a 16-slot least-frequently-used cache, filling cache misses with bounded parallel read operations into Metal-visible buffers. Metal computes the shared-expert branch while these reads occur in parallel, then combines shared and routed outputs. Prompt prefill uses chunks of up to 128 tokens to allow a single fetched expert to serve multiple rows, while generation repeats the routed layer loop one token at a time.
The model installation process applies the same bounded-memory principle. The streaming installer never materializes the full source checkpoint, instead streaming required byte ranges from a pinned Hugging Face revision and repacking them directly into the .gturbo layout as they arrive. This approach avoids requiring a second full checkpoint on disk and keeps scratch memory bounded during installation. The completed installation occupies approximately 14.3 GB and is validated through manifest and file hash verification before use.
TurboFieldfare requires macOS 26 with Metal 4, Xcode 26, and Swift 6.2 or newer, targeting arm64 Apple Silicon exclusively. The project includes comprehensive documentation covering system design, optimization experiments with 103 measured results across kernels, caching, I/O, and decode operations, implementation references, runtime controls, and benchmarking methodology. Generation supports configurable temperature, Top-K, and Top-P sampling, with temperature set to zero enabling deterministic greedy output. The system is text-only, supporting user and model messages with optional system guidance but not exposing tools, images, audio, or video.
