The oobabooga/textgen repository provides an open-source desktop application designed to run large language models (LLMs) locally on a user's machine. Its primary goal is to offer a private, offline, and user-friendly interface for interacting with LLMs, supporting both text and vision (multimodal) tasks. The application is compatible with Windows, macOS, and Linux, and offers portable builds that include all necessary dependencies, making installation and setup straightforward for users of varying technical backgrounds.
TextGen supports a wide range of LLM backends, including llama.cpp, ik_llama.cpp, Hugging Face Transformers, ExLlamaV3, and NVIDIA TensorRT-LLM, allowing users to switch between different models and backends without restarting the application. It is compatible with GGUF (llama.cpp) model files, and also supports other model formats such as Transformers and EXL3, which require a one-click installer. The repository provides detailed instructions for downloading and placing models, ensuring that the application automatically detects and loads them.
The user interface is designed for both chat and text generation workflows. It features multiple chat modes, including instruction-following (similar to ChatGPT) and character-based conversations. Prompts are automatically formatted using Jinja2 templates, and users can attach images to messages for vision-enabled models. The app also supports uploading and discussing the contents of text files, PDFs, and Word documents. Advanced conversation management features include editing messages, navigating between message versions, and branching conversations. A notebook tab is available for free-form text generation outside the chat paradigm.
TextGen includes a robust API that is compatible with OpenAI and Anthropic endpoints, supporting chat, completions, and tool-calling. This allows the application to serve as a local drop-in replacement for cloud-based LLM APIs, enabling integration with existing tools and workflows. Tool-calling functionality lets models invoke custom Python functions during conversations, supporting tasks like web search, page fetching, and mathematical calculations. The system is extensible, with support for both built-in and community-developed extensions such as text-to-speech, voice input, and translation.
For users interested in model customization, TextGen offers training capabilities, including fine-tuning LoRA adapters on multi-turn chat or raw text datasets. Training sessions can be resumed if interrupted. The application also features an image generation tab, supporting diffusers models like Z-Image-Turbo, with options for quantization and a persistent gallery for generated images and metadata.
Privacy is a core focus of TextGen. The application operates entirely offline, with no telemetry, external resource fetching, or remote update requests. The interface supports dark and light themes, syntax highlighting for code, and LaTeX rendering for mathematical expressions. Installation options are flexible, ranging from portable builds and manual Python virtual environments to full-featured Conda environments and Docker containers, catering to a wide range of hardware and user preferences. Comprehensive documentation and tutorials are provided to guide users through setup, model management, training, and extension development.
In summary, oobabooga/textgen is a comprehensive, privacy-focused desktop solution for running and interacting with local LLMs, offering advanced features for chat, vision, tool integration, training, and extensibility, all within an easy-to-use and highly configurable environment.
