TextGen is an open-source desktop application designed for running large language models locally on personal computers. Written in Python, it provides a complete interface for text generation, vision capabilities, tool-calling, and API access while maintaining complete privacy through offline operation with zero telemetry. The application is available as portable builds for Linux, Windows, and macOS with multiple hardware acceleration options including CUDA, Vulkan, ROCm, and CPU-only configurations.
The core functionality spans multiple interaction modes. Users can operate in instruct mode for instruction-following similar to ChatGPT, chat-instruct mode, or standard chat mode for conversations with custom characters. The application automatically formats prompts using Jinja2 templates. Vision capabilities allow users to attach images to messages for visual understanding, while file attachment features support text files, PDF documents, and .docx files for content discussion. The interface enables message editing, navigation between message versions, and conversation branching at any point. A separate notebook tab provides free-form text generation outside of structured chat turns.
TextGen supports multiple inference backends including llama.cpp, ik_llama.cpp, Transformers, ExLlamaV3, and TensorRT-LLM, with the ability to switch between backends and models without restarting the application. The application provides OpenAI and Anthropic-compatible API endpoints for chat, completions, and messages, enabling it to function as a local drop-in replacement for commercial APIs. Tool-calling functionality allows models to invoke custom functions during conversations, including web search, page fetching, and mathematical operations, with tools implemented as single Python files. The system also supports MCP servers for extended functionality.
Additional capabilities include LoRA fine-tuning on multi-turn chat or raw text datasets with support for resuming interrupted training runs. An image generation tab dedicated to diffusers models like Z-Image-Turbo offers 4-bit and 8-bit quantization options alongside a persistent gallery with image metadata. The interface features dark and light themes, syntax highlighting for code blocks, and LaTeX rendering for mathematical expressions. Community and built-in extensions provide text-to-speech, voice input, and translation functionality.
Model loading is straightforward, with GGUF format models from Hugging Face placed directly into the user_data/models folder for automatic detection. The application includes installation options ranging from portable builds requiring no setup to full installations supporting additional backends, training, and extensions. Installation scripts for Windows, Linux, and macOS automate environment setup through Miniforge-based Conda environments, with Docker support also available for containerized deployment.
