Moshi is a speech-text foundation model and full-duplex spoken dialogue framework that enables real-time two-way conversation with simultaneous speech input and output.
The tool addresses the challenge of building interactive spoken dialogue systems by modeling two independent audio streams—one for the user's speech and one for Moshi's response—alongside text token predictions representing Moshi's inner monologue. This dual-stream approach with internal text modeling improves generation quality. The system uses Mimi, a streaming neural audio codec that compresses 24 kHz audio to 1.1 kbps bandwidth while maintaining low latency, achieving theoretical latency of 160ms and practical latency as low as 200ms on an L4 GPU.
Developers should choose this tool if they need to build real-time spoken dialogue applications where simultaneous bidirectional speech is essential. The repository provides three separate inference implementations suited to different deployment contexts: a PyTorch version for research and experimentation, an MLX implementation for on-device inference on macOS and iPhone, and a production-ready Rust implementation with Python bindings. The Rust stack includes a native Mimi codec implementation. The project also provides a web UI client used in the live demo. For fine-tuning, developers are directed to a separate repository.
The project maintains active continuous integration workflows for both Python and Rust code quality. Development activity spans multiple implementation languages and deployment targets, indicating ongoing work to support research prototyping, edge deployment, and production use cases simultaneously.
