lm-sys/FastChat

FastChat is an open platform for training, serving, and evaluating large language model based chatbots.

FastChat addresses the need for a unified system to build, deploy, and benchmark LLM chatbots. It provides training and evaluation code for models like Vicuna and MT-Bench, a distributed multi-model serving system with web UI and OpenAI-compatible REST APIs, and infrastructure for comparative evaluation through side-by-side LLM battles. The platform powers Chatbot Arena, which collects human preference votes to maintain an LLM Elo leaderboard.

Developers should choose FastChat if they need to serve multiple LLMs simultaneously with a unified interface, conduct comparative evaluation of models, or build on established benchmarking infrastructure. It suits projects ranging from research evaluation to production chatbot services. The tool is particularly valuable for teams wanting to replicate or extend Chatbot Arena's evaluation methodology or deploy models like Vicuna that are specifically trained and optimized within this ecosystem.

The project maintains active development with regular releases of new model variants and datasets. It has published technical reports documenting its evaluation methodology and released large-scale conversation datasets for research. The codebase includes comprehensive tooling for both inference and fine-tuning workflows, with support for various context lengths and model architectures. Development activity shows ongoing refinement of the serving infrastructure and expansion of supported models and evaluation benchmarks.