ulab-uiuc/LLMRouter

LLMRouter is a Python library for intelligent routing that dynamically selects the most suitable language model for each query based on task complexity, cost, and performance requirements.

The tool addresses the problem of inefficient LLM inference by implementing smart routing logic that directs queries to optimal models rather than using a single model for all requests. It supports over sixteen routing models organized into five categories: single-round routers, multi-round routers, multimodal routers, agentic routers, and personalized routers. These routers employ diverse strategies including KNN, SVM, MLP, Matrix Factorization, Elo Rating, graph-based approaches, BERT-based methods, and hybrid probabilistic techniques. The library provides a unified command-line interface for training and inference, along with a Gradio-based interactive chat interface. It also includes a data generation pipeline that can create training data from multiple benchmark datasets with automatic API calling and evaluation.

Developers should consider this tool if they operate multiple language models and want to optimize inference efficiency and cost. It suits projects where query complexity varies significantly and where routing decisions can meaningfully reduce latency or expense. The project is particularly relevant for teams building multi-model systems or exploring different routing strategies without implementing each from scratch.

The project maintains active engagement with its community through multiple communication channels and welcomes external contributions. Development activity shows ongoing refinement of the routing framework and expansion of supported routing strategies. The team has published research backing the approach and continues to integrate new routing models and capabilities into the library.