Tract is a neural-network inference engine that loads ONNX and NNEF models, optimizes them, and executes them across diverse hardware platforms.
The tool addresses the challenge of deploying trained neural networks efficiently across heterogeneous environments. It accepts ONNX and TensorFlow Lite models as input, applies optimization passes to reduce computational overhead, and generates executable inference code. A distinctive feature is its NNEF-based intermediate format, which enables a translate-once, ship-tiny-runtime workflow: models are optimized once and the resulting compact runtime can be deployed to resource-constrained environments without requiring the full inference engine.
Tract suits projects requiring neural-network inference on embedded systems, mobile devices, or edge hardware where model size and latency matter. It runs on ARM CPUs, NVIDIA and Apple GPUs, WebAssembly in browsers, and standard workstations. The project is particularly strong for speech recognition and wake-word detection, having been battle-tested in production at Sonos for these workloads. It also handles LLM inference, text-to-image generation, and classical computer vision tasks. The codebase provides runnable examples covering mobile vision models, transformer-based text generation, streaming speech recognition, object detection, and in-browser execution, giving prospective users concrete reference implementations.
The project maintains active test coverage across native Linux and embedded target platforms. Development includes regular updates to the core inference engine and API layers, with both Rust and Python interfaces actively maintained. The tool receives ongoing refinement to support contemporary model architectures and optimization techniques.
