KrillinAI is a comprehensive video translation and dubbing tool designed to serve both human users and AI Agents, built primarily in Go and supporting over 100 languages. The project provides a complete end-to-end pipeline for video localization that encompasses downloading videos, transcribing speech, translating content, generating text-to-speech dubbing, reformatting videos, and creating cover images. The tool is optimized for major platforms including YouTube, TikTok, Bilibili, Douyin, Xiaohongshu, and WeChat Video, handling both landscape and portrait video formats to ensure content displays correctly across different social media channels.
The repository's core functionality is organized around a modular architecture that allows both integrated one-click processing and independent invocation of individual stages through a command-line interface. Users can execute stages like subtitle generation, TTS dubbing, horizontal rendering, vertical rendering, and cover generation either sequentially through a pipeline command or separately as needed. Each stage outputs structured JSON results and writes artifacts to a manifest file, enabling subsequent stages to reuse prior outputs. This design makes the tool particularly suitable for automation workflows and AI Agent orchestration, where different processing stages can be chained together on demand.
For speech recognition, KrillinAI supports multiple services including OpenAI Whisper, FasterWhisper, WhisperKit for macOS M-series chips, WhisperCpp, and Alibaba Cloud ASR. The tool offers both cloud-based and local model options, with local models supporting automatic installation of executable files and model weights. For large language model integration, the system is compatible with any service following OpenAI API specifications, including OpenAI, Gemini, DeepSeek, Tongyi Qianwen, and locally deployed open-source models. Text-to-speech capabilities are provided through Alibaba Cloud Voice Service, OpenAI TTS, and MiniMax TTS, with voice cloning options available through CosyVoice or custom voice implementations.
The repository is classified across multiple domains including NLP applications, text generation, data processing, machine learning tools, and AI research, reflecting its sophisticated use of language models and audio processing technologies.
The tool is available in multiple deployment modes including a desktop application for end users, a web-based interface for server deployment, and command-line tools for automation. The project provides releases for Windows, Linux, and macOS, with specific handling for macOS users regarding application signing. Docker deployment is supported for containerized environments. Documentation is available in multiple languages including English, Simplified Chinese, Japanese, Korean, Vietnamese, French, German, Spanish, Portuguese, Russian, and Arabic, indicating broad international reach and community support.
