Mesh LLM is a distributed inference platform that pools GPU and memory resources across machines and exposes them through an OpenAI-compatible API.
The tool solves the problem of running large language models when no single machine has sufficient resources. It aggregates compute across a network of nodes and automatically decides whether to run a model locally, route it to a peer node, or split it across multiple machines using Skippy stage splits for models too large for any individual box. Users start with a single node and add more nodes later; the mesh handles routing and resource allocation transparently.
Mesh LLM suits developers and organizations wanting to share GPU compute privately or publicly without centralized infrastructure. It works for teams running agents and chat applications who have multiple machines with spare capacity, or for anyone wanting to experiment with larger models than their hardware alone supports. The tool provides several deployment modes: joining a public mesh, creating a private mesh, running locally without networking, or publishing your own mesh. It also supports specialized workflows like running models with layer splits, attaching SSD-backed backends, and fanning prompts across all models in a mesh for ensemble inference. Built-in agent support includes integrations with Goose, OpenCode, Claude Code, and Pi.
The project shows consistent development activity with regular commits across core infrastructure and documentation. Work spans distributed networking logic, model serving capabilities, and the web console interface. The maintainers actively document workflows and troubleshooting guides, including platform-specific installation instructions and detailed guides for mesh networking, model splitting, and agent usage. Contributions are welcomed through a documented build process.
