LiteLLM review

LiteLLM is a self-hosted AI gateway, not just a hosted API or a model provider. It sits between your applications and model, agent and MCP endpoints, giving clients one OpenAI-compatible API and key to use across providers. Its job is to route requests, control access, and collect usage and spend data.

For a platform team, that puts LiteLLM in the same operational path as an API gateway or internal service proxy. The homepage says teams can use it with internal, fine-tuned and self-hosted models, as well as external providers. It also describes routing, governance, observability and optimization as gateway functions. Teams can scope access by team, project or app, and set budgets and rate limits; requests stop when caps are reached, according to the product facts.

The trade-off is operational ownership: self-hosting means your team runs the gateway and manages its deployment. LiteLLM says it supports 140+ providers and 1,800+ models.

The main integration surface is an OpenAI-compatible API. The vendor says it covers 140+ providers and 1,800+ models, with model swaps handled in gateway configuration rather than app code. That makes it a fit for applications already using an OpenAI-style request pattern, though provider-specific features and behavior still need validation against the docs.

LiteLLM names a Python SDK for calling 100+ LLMs in the OpenAI format. It also offers a liteagents SDK, described as an agent SDK that routes each turn to an appropriate model across providers. The homepage links to AI Gateway docs and provides a quick-start installation option for macOS, Linux and Windows; its shell command downloads and runs a script. Confirm current installation steps and SDK support in the vendor's docs before standardizing on them.

The gateway also exposes spend tracking, key management and access controls through its product interface. The homepage does not name SDKs for other languages, so teams using another language should verify their client approach and supported API behavior in the docs.

The Open Source plan costs $0 per month and is listed as free forever. It includes one OpenAI API for 100+ providers, virtual keys, users and teams, spend tracking, budgets, rate limits, fallbacks, request logging and Prometheus metrics. That is a useful starting point for teams willing to host and operate the gateway themselves.

Enterprise is quoted annually. The pricing details say the quote depends on gateway request capacity, deployment architecture and support needs. Its listed features include SSO, SCIM, OIDC/JWT authentication, audit logs, secret managers, a multi-region control plane, and optional 24/7 support and SLAs. Air-gapped deployment is also available. A 30-day free trial is listed, but confirm which plan it applies to before budgeting around it.

The vendor doesn't publish a numerical request allowance, overage price or rate-limit tier for these plans. Open Source has no listed LiteLLM subscription fee, but self-hosting requires infrastructure and model providers may charge separately for inference. Ask Enterprise sales how capacity is measured and what happens when it is exceeded.

LiteLLM describes itself as open source and self-hosted, and its quick-start command points to a public GitHub repository. The homepage does not identify the repository's licence; check the licence text and distribution terms before embedding it in a commercial product or modifying it for internal use.

The vendor's main lock-in reduction is the OpenAI-compatible API and the ability to swap models through gateway configuration without changing app code. That can reduce application-level migration work, but it does not guarantee that provider-specific parameters, tool calls or response behavior will transfer unchanged. A commenter on r/Python also noted that provider edge cases matter regardless of which abstraction layer a team uses.

The homepage interface shows an Export control, but it does not establish which data that exports. Before adoption, confirm how to export spend records, configuration and logs, and test a route that bypasses the gateway. That gives you a practical exit path if you later replace the proxy.

LiteLLM offers production controls such as budgets, rate limits, SSO, audit logs and secret-manager integration, with some governance features listed under Enterprise. The vendor doesn't publish numerical gateway request limits or capacity tiers, so ask about throughput, scaling behavior and overages before sizing a deployment. Also confirm how caps behave during provider failures and how logs handle sensitive prompts and credentials.

Release security deserves specific review. Reddit users reported that LiteLLM 1.107.0 on PyPI and official Docker images carried credential-stealing malware and a backdoor; another thread reported compromised 1.82.7 and 1.82.8 packages. These are user reports, not a substitute for a vendor incident statement, but they describe a material risk for a gateway that holds provider credentials. Pin exact versions, verify package and image provenance, and define a credential-rotation process.

A Reddit commenter recommended pinning versions and checking hashes in CI. Treat upgrades as controlled production changes rather than allowing floating image tags to update automatically.