mnfst/llm-gateway

LLM Gateway is a gateway and routing layer that connects AI agents and applications with any language model provider.

The tool solves the problem of managing multiple LLM provider integrations and controlling how requests flow to different models. It acts as a centralized gateway that abstracts away provider-specific details, allowing agents and applications to send requests through a single interface regardless of which underlying provider handles them. The gateway handles routing logic, enabling developers to direct requests to different providers based on configurable rules.

Teams building AI agents should consider this tool if they need to work with multiple LLM providers, track token usage and costs across different models, or want to avoid vendor lock-in by maintaining flexibility in provider selection. It suits projects where observability into LLM usage is important and where the ability to switch or balance load across providers adds value. The tool supports bring-your-own-key deployments, meaning teams can use their own API credentials rather than relying on a managed service.

The project maintains active continuous integration with automated testing on code changes. Code coverage is tracked and monitored as part of the development process. The tool is available as a Docker image for containerized deployment and supports deployment across multiple cloud platforms through provided infrastructure templates.