AudioCraft is a PyTorch library for deep learning research on audio generation and processing. The library provides inference and training code for multiple state-of-the-art generative models that produce high-quality audio, centered around two primary capabilities: MusicGen for controllable text-to-music generation and AudioGen for text-to-sound generation. It also includes EnCodec, a neural audio codec for high-fidelity compression and tokenization.
The library addresses the need for accessible, research-grade tools to work with audio generation models. AudioCraft bundles several complementary components: MusicGen and AudioGen handle the core generation tasks with textual conditioning, EnCodec provides efficient audio compression with neural encoding, Multi Band Diffusion offers an alternative decoder for EnCodec using diffusion techniques, MAGNeT provides non-autoregressive generation, AudioSeal handles audio watermarking, MusicGen Style adds style-based conditioning to music generation, and JASCO enables generation conditioned on chords, melodies, and drum tracks. The library is built on PyTorch and requires Python 3.9 and PyTorch 2.1.0, with ffmpeg recommended for audio processing.
Developers should adopt AudioCraft if they need to conduct research on audio generation or integrate state-of-the-art audio synthesis into applications. The library suits projects requiring controllable music or sound generation with various conditioning options, from simple text prompts to melodic and stylistic guidance. It is particularly valuable for those who want access to training code, not just inference—the README confirms that training pipelines are available for EnCodec, MusicGen, Multi Band Diffusion, and JASCO, enabling researchers to reproduce existing work or develop custom models. The library provides API documentation and model caching through Hugging Face integration.
The project maintains automated testing, linting, and documentation workflows. Training code is actively documented with model-specific configuration guides and FAQs. The library includes comprehensive PyTorch components designed for deep learning research, with training pipelines that support custom development beyond the provided models.
