The exo repository is a Python-based project that enables users to run frontier AI models locally by connecting multiple devices into a distributed AI cluster. Maintained by exo labs, the project has grown to 45,930 stargazers as of July 2026, with consistent community engagement reflected in its active issue and pull request tracking.

The core functionality of exo centers on distributed inference across heterogeneous devices. The system automatically discovers devices on the network without requiring manual configuration, eliminating setup friction for users wanting to pool computational resources. A key differentiator is day-0 support for RDMA over Thunderbolt 5, which the project claims enables a 99 percent reduction in latency between devices. This capability allows models to run faster as additional devices are added to the cluster, addressing a fundamental challenge in distributed AI inference where communication overhead typically limits scaling benefits.

The project implements topology-aware automatic parallelization, meaning exo analyzes the real-time device topology including available resources and network characteristics between devices to determine optimal model distribution strategies. The system supports tensor parallelism with reported speedups of up to 1.8x on two devices and 3.2x on four devices. The inference backend uses MLX and MLX distributed for distributed communication, providing a foundation for efficient computation across the cluster.

API compatibility is a significant design consideration. exo supports multiple API standards including OpenAI Chat Completions API, Claude Messages API, OpenAI Responses API, and Ollama API, allowing users to leverage existing tools and clients without modification. The project also supports loading custom models from the HuggingFace hub, expanding the range of available models beyond built-in options.

A built-in dashboard provides cluster management and direct model interaction capabilities, accessible at localhost:52415. The project includes benchmark data demonstrating performance on multi-device setups, including configurations running models like Qwen3-235B and DeepSeek v3.1 671B on clusters of M3 Ultra Mac Studios.

The repository shows active maintenance with a median issue and pull request response latency of 1.4 hours across 892 tracked items, indicating responsive community engagement. The most active contributors tracked by GitGenius are Evanev7 with 951 events, AlexCheema with 653 events, and rltakashige with 342 events. Bug reports and enhancement requests represent the most common issue types with 158 and 126 respectively. The project maintains connections with major technology repositories including microsoft/vscode, microsoft/typescript, and rust-lang/rust through overlapping contributors.

Installation options span macOS and Linux platforms, with a native macOS app available for background operation. The project follows the XDG Base Directory Specification on Linux for configuration and data storage. Current limitations include CPU-only execution on Linux with GPU support under development, while macOS benefits from Metal acceleration on Apple Silicon. The system supports coordinator-only nodes for machines with good network connectivity but limited computational resources, and includes legacy daemon mode for compatibility with older init systems.