LiteLLM is an open source AI Gateway written in Python that provides a unified interface for calling over 100 different large language model providers through a single standardized API format compatible with OpenAI. The project is maintained by BerriAI, a Y Combinator W23 company, and serves as both a Python SDK for direct library integration and a deployable proxy server for centralized LLM access across organizations.
The core problem LiteLLM solves is the fragmentation in the LLM ecosystem. Different providers like OpenAI, Anthropic, Bedrock, Azure, Vertex AI, Cohere, and others each have their own SDKs, authentication patterns, request formats, and error handling mechanisms. LiteLLM abstracts away these differences by presenting a unified OpenAI-compatible interface, allowing developers to swap between providers without rewriting application code. The gateway supports multiple deployment options including direct deployment to Render, Railway, AWS, and GCP through one-click deployment buttons.
As a proxy server and AI gateway, LiteLLM includes production-ready features for enterprise use. These encompass virtual key management, cost tracking and spend monitoring, guardrails for safety and compliance, load balancing across multiple models and providers, comprehensive logging, and an admin dashboard. The gateway achieves 8 milliseconds P95 latency at 1000 requests per second according to published benchmarks, making it suitable for high-throughput production environments.
Beyond basic LLM calls, LiteLLM supports multiple endpoint types including chat completions, embeddings, image generation, audio processing, batch operations, reranking, and agent-to-agent communication. The platform integrates with agent frameworks including LangGraph, Vertex AI Agent Engine, Azure AI Foundry, Bedrock AgentCore, and Pydantic AI through an A2A protocol. Additionally, LiteLLM functions as an MCP gateway, allowing Model Context Protocol servers to be connected to any LLM through the gateway's chat completions endpoint, with specific integration support for Cursor IDE.
The repository shows active development and community engagement.
The project has achieved notable adoption among major technology companies including Stripe, Google, Netflix, and OpenAI, indicating its viability as an enterprise-grade solution. LiteLLM is available on PyPI as a Python package and maintains comprehensive documentation at docs.litellm.ai, with additional resources including Discord and Slack communities for user support. The codebase overlaps with contributors from major projects including Microsoft's VSCode and TypeScript repositories, as well as the Rust language project, suggesting cross-pollination with the broader developer ecosystem.
