Miso TTS is a text-to-speech model that generates conversational speech from text with optional audio conditioning for voice cloning.
The model addresses the challenge of producing emotionally expressive, natural-sounding speech in dialogue contexts. It uses a two-stage transformer architecture combining a large backbone transformer that processes interleaved text and audio embeddings with a smaller autoregressive decoder that predicts higher-order audio codebooks. This design allows the model to condition generation on conversation history, enabling contextually appropriate speech synthesis. The system tokenizes audio using the Mimi codec and represents speech as discrete codes across multiple codebooks, which the model learns to predict autoregressively.
Miso TTS suits developers building conversational AI systems, voice applications, or tools requiring natural speech synthesis with emotional nuance. The model currently supports English only. It offers voice cloning capabilities by accepting prior audio as optional context, allowing users to maintain consistent speaker characteristics across generated utterances. The tool is straightforward to adopt, with inference code and model weights publicly available; the model downloads automatically on first use through Hugging Face Hub and caches locally for subsequent runs. A demo is available on the project's landing page for immediate experimentation before local deployment.
The project maintains active engagement with the community through multiple channels including a website, social media presence, and Hugging Face model hosting. Development focuses on providing clear documentation with quickstart instructions, example scripts, and architectural explanations through blog posts. The codebase includes safety considerations as a documented section of the repository.
