MLX-Audio is a Python-based audio processing library built on Apple's MLX framework, designed to deliver fast and efficient speech synthesis, recognition, and conversion specifically optimized for Apple Silicon devices. The library provides implementations of text-to-speech (TTS), speech-to-text (STT), and speech-to-speech (STS) capabilities, making it a comprehensive solution for audio tasks on M-series chips.
The repository supports an extensive collection of models across all three primary audio domains. For text-to-speech, it includes over twenty different model architectures ranging from lightweight options like KittenTTS and Soprano to advanced multimodal systems like Ming Omni TTS and KugelAudio. These TTS models support anywhere from single languages to over 600 languages, with features including voice cloning, style control, and adjustable speech speed. The speech-to-text implementations feature models from major organizations including OpenAI's Whisper, Alibaba's Qwen3-ASR, NVIDIA's Parakeet and Nemotron systems, and Meta's massively multilingual MMS supporting over 1000 languages. STT models in the library offer capabilities like speaker diarization, word-level alignment, streaming inference, and language identification.
The library provides multiple interfaces for users. A command-line interface allows straightforward audio generation and processing with options for streaming output and audio joining. A Python API enables programmatic access to all functionality. The repository includes an interactive web interface with 3D audio visualization and an OpenAI-compatible REST API for integration into existing systems. Installation is available through pip or uv, with separate options for command-line tools, full development environments, and web interface support.
Performance optimization is central to MLX-Audio's design. The library supports quantization at multiple bit depths including 3-bit, 4-bit, 6-bit, and 8-bit formats to reduce model size and improve inference speed on Apple Silicon. A Swift package is available for iOS and macOS integration, extending the library's reach to Apple's native platforms.
Bug reports represent the most active issue category with nine tracked items, followed by enhancement requests with five items. The repository shares contributors with several other significant projects including ollama/ollama, unslothai/unsloth, and ggml-org/llama.cpp, indicating active participation in the broader machine learning and inference optimization ecosystem.
