HeartMuLa is a family of open-source music foundation models that generates music conditioned on lyrics and tags with multilingual support.
The project addresses music generation by combining several specialized components. HeartMuLa itself is a music language model that accepts lyrics and tags as conditioning inputs and produces music across nearly all languages. The approach is grounded in a codec-based architecture: HeartCodec provides a 12.5 Hz music codec for high-fidelity audio tokenization and reconstruction, enabling discrete token-based generation. Supporting this core capability are HeartTranscriptor, a lyrics transcription model tuned from Whisper, and HeartCLAP, an audio-text alignment model that creates a unified embedding space for music descriptions and cross-modal retrieval.
Developers should adopt this tool if they need open-source music generation with strong multilingual capabilities and want to avoid proprietary solutions. The project suits research applications, music information retrieval tasks, and systems requiring controllable generation from text descriptions. The toolkit also includes MuLaCover, a specialized model for cover-song and music-remix generation that preserves musical identity through symbolic melody and harmony while allowing genre, instrumentation, and mood modifications via text.
Development shows sustained activity with recent releases of both encoder and decoder components for HeartCodec, enabling practical audio tokenization workflows. The project maintains active community engagement through Discord and provides comprehensive documentation including usage examples and generation guides. Model weights are distributed across multiple platforms alongside code repositories, indicating a commitment to accessibility for the research community.
