We may earn a commission if you buy through links on this page. It never changes a score or a ranking. How we make money
An LLM gateway gives an application one integration point for models from multiple providers. OpenRouter is our best overall pick for teams that want provider choice and failover; Vercel AI Gateway fits teams already building with the AI SDK. For self-hosting, compare LiteLLM, Portkey and Helicone, but assess each project’s operational and maintenance risks. Prices range from free plans to listed starting prices of $79, and several vendors do not publish a starting price. A free tier alone does not tell you what production traffic will cost, so verify model charges, gateway fees and limits before choosing.
Quick picksThe shortlist
OpenRouterOpenRouterBest overall · Free
LiteLLMLiteLLMBest open source · Free
Vercel AI GatewayVercel AI GatewayBest for AI SDK · Free
PortkeyPortkeyBest for private deployment · $49/mo
RequestyRequestyBest for fallback routing · Free
Eden AIEden AIBest for EU controls · Free
Cloudflare AI GatewayCloudflare AI GatewayBest for Cloudflare teams · Free
HeliconeHeliconeBest for LLM observability · $79/mo
YepAPIYepAPIBest for SEO and AI · Custom
NanoGPTNanoGPTBest for multimodal prototypes · $12/mo
ZenMuxZenMuxBest for cost monitoring · Custom
01Compared at a glance
| Tool | Score | From | Free plan | Trial |
|---|---|---|---|---|
| OpenRouterOpenRouter | 7.6 | Free | Yes | — |
| LiteLLMLiteLLM | 7.1 | Free | Yes | 30 days |
| Vercel AI GatewayVercel AI Gateway | 7.8 | Free | Yes | — |
| PortkeyPortkey | 7.6 | $49/mo | Yes | — |
| RequestyRequesty | 7.5 | Free | Yes | — |
| Eden AIEden AI | 7.2 | Free | Yes | — |
| Cloudflare AI GatewayCloudflare AI Gateway | 7.0 | Free | Yes | — |
| HeliconeHelicone | 6.8 | $79/mo | Yes | 7 days |
| YepAPIYepAPI | 7.2 | Custom | No | Yes |
| NanoGPTNanoGPT | 6.5 | $12/mo | Yes | — |
| ZenMuxZenMux | 6.2 | Custom | No | — |
02The picks, in detail
- Best overallOpenRouterOpenRouter
OpenRouter puts models from multiple providers behind one API and is aimed at teams evaluating models, consolidating billing or needing provider failover. That makes it a strong general-purpose starting point when provider choice matters more than deep platform-specific integration. Its free plan is listed, but the starting price is not published; check routing behavior, limits and production costs before relying on failover.
What works
- One API reaches 600+ models from 80+ providers
- Automatic routing uses live marketplace data
- Failover is on by default
What doesn't
- Standard and Business charge fees on credit purchases
- Free plan is limited to 50 requests per day
- Provider speed and caching can vary
- Best open sourceLiteLLMLiteLLM
LiteLLM is a free, open-source, self-hostable gateway for teams managing multiple model providers and shared access controls. It suits platform teams that want to operate the gateway themselves rather than depend on a hosted service. The trade-off is operational ownership: pin releases and assess supply-chain risk before storing production credentials. Its starting price is not published.
What works
- One OpenAI-compatible API for 140+ providers
- Routes requests without app-code model changes
- Tracks spend across keys, teams and agents
What doesn't
- Self-hosting adds infrastructure and operations work
- Enterprise pricing is custom and annual
- Numerical gateway limits are not published
- Best for AI SDKVercel AI GatewayVercel AI Gateway
Vercel AI Gateway is a hosted model-routing API for teams integrating multiple providers, especially developers already using the AI SDK. Provider failover can reduce custom integration work, but verify how routing behaves for your models and workloads before production. A free plan is listed; the starting price is not published, so calculate actual usage costs and check service limits.
What works
- One API key and endpoint for 400+ models
- Provider-price token billing, with no platform fee
- Automatic fallback to another provider
What doesn't
- Free tier is limited to eligible models
- Free tier has lower per-model rate limits
- Paid usage requires purchased credits
- Best for private deploymentPortkeyPortkey
Portkey centralizes access to models across providers, with logs and controls for application and enterprise teams. It is open source and self-hostable, making it worth evaluating when private deployment matters. The listed starting price is $49, with a free plan also listed. Confirm which deployment and production features are included at the price you need.
What works
- Unified API provides access to 1,600+ LLMs
- Routing supports fallbacks, load balancing and retries
- Logs, traces and usage are available for review
What doesn't
- Homepage names no SDKs or programming languages
- Developer plan allows only 10,000 recorded logs monthly
- Production overages are metered by requests, not logs
- Best for fallback routingRequestyRequesty
Requesty is a hosted gateway for routing across model providers, with fallback intended to avoid building all retry logic into an application. It also targets organizations centralizing AI spend controls. A free plan is listed, but the starting price is not published. Before adoption, ask about request limits, routing controls and how usage charges are calculated.
What works
- OpenAI-compatible API and familiar SDK integration
- Routing policies cover latency, geography and traffic shares
- Automatic failover handles provider errors and rate limits
What doesn't
- Pay as you go adds a 5% model-spend markup
- Free plan is limited to 200 requests per day
- Gateway rate limits are not published
- Best for EU controlsEden AIEden AI
Eden AI is a hosted, OpenAI-compatible gateway for routing across AI providers, tracking per-request costs and managing access. Its listed starting price is $0, and it targets organizations requiring EU data controls. The trade-off for teams that need deployment ownership is that the vendor does not publish a self-hosting path. Confirm data handling and usage limits for your workload.
What works
- One OpenAI-compatible API covers around 1,000 models
- Request responses include cost information
- Route by cost, speed or latency
What doesn't
- Self-serve is not free usage; provider usage still costs
- Credit purchases add a 5.5% fee
- Default rate limit is 10 requests per second
- Best for Cloudflare teamsCloudflare AI GatewayCloudflare AI Gateway
Cloudflare AI Gateway is a hosted option for teams already using Cloudflare that need multi-provider routing, caching and request-level logs. Those controls can make it a fit when AI traffic should sit alongside existing Cloudflare operations. A free plan is listed; the starting price is not published. Confirm pricing, limits and which providers and caching behaviors your application needs.
What works
- Routes requests by latency, cost or availability
- Caches frequent responses to reduce repeat calls
- Logs prompts, responses, costs and token usage
What doesn't
- AI Gateway-specific prices and billing terms are not published
- Rate-limit thresholds are not published
- Source availability and license are not published
- Best for LLM observabilityHeliconeHelicone
Helicone combines LLM routing and observability, including request-level cost visibility. It is open source and self-hostable, which may suit teams that want to operate the service themselves. The listed starting price is $79, with a free plan also listed. Developers have reported maintenance-mode risk, so check the project's current status and support expectations before adopting it.
What works
- OpenAI-compatible proxy and asynchronous logging
- One API key for access to 100+ models
- Tracks latency, tokens, errors and costs
What doesn't
- Reddit users report a move toward maintenance mode
- Usage and storage overage rates are not published
- Hobby ingestion is limited to 10 logs per minute
- Best for SEO and AIYepAPIYepAPI
YepAPI combines SEO, scraping, social and e-commerce data APIs with access to 285 AI models under one usage-billed API. It fits applications that need those data services alongside AI calls, or prototypes built around metered API usage. It is less clearly focused on gateway operations than dedicated routing products, so confirm provider routing, fallback behavior and request controls for production.
What works
- One key covers SEO, scraping, social data and AI models
- Consistent JSON response envelope across endpoints and models
- $5 signup credit with no card required
What doesn't
- SDKs and supported programming languages are not named
- No published self-hosting option or open-source licence
- Data export and migration tooling are not published
- Best for multimodal prototypesNanoGPTNanoGPT
NanoGPT offers a multi-model gateway for developers using compatible API clients, with chat and media generation. It can suit prototyping across providers, but reports raise questions about provider routing and reliability in production. The listed starting price is $12, with a free plan also listed. Validate the models and failure behavior your application depends on before committing.
What works
- Catalog lists 1,277 models, including ChatGPT, Claude and Gemini
- OpenAI-compatible API offers a familiar integration surface
- First web chat is free and requires no account
What doesn't
- Provider routing can cause slow or inconsistent responses
- Vendor describes rate limits as generous but gives no numeric caps
- SDKs and supported programming languages are not named
- Best for cost monitoringZenMuxZenMux
ZenMux provides one API for multiple model providers and targets teams that want provider failover plus token and request cost monitoring. That makes it worth considering when usage visibility is a central requirement. Its starting price is not published, so ask for a production cost estimate, service limits and details of routing behavior before choosing it.
What works
- One account and API for multiple model providers
- Supports OpenAI, Anthropic and Google Vertex AI protocols
- GUI covers chat, image and video generation
What doesn't
- Per-call prices and a starting price aren't published
- No general free plan is listed
- No SDK languages are named on the homepage
03How we chose
We ranked these products for developer and platform-team fit: whether they centralize access across providers, offer routing or failover, and provide cost or request controls. We also considered API compatibility, named SDK fit, documentation needs, free plans, listed starting prices, open-source licensing and self-hosting. Where production-risk reports are noted, we treat them as signals to investigate rather than proof of failure: LiteLLM has a supply-chain risk caveat, Helicone has maintenance-mode concerns, and NanoGPT has routing and reliability reports. We did not claim hands-on testing or benchmarks.
04Check the integration and routing path
Start with the request path your application will use. Confirm whether the gateway accepts the API format your code already sends, whether it has an SDK for your language, and whether the docs cover authentication, streaming, errors and model selection. Eden AI is described as OpenAI-compatible, NanoGPT supports compatible API clients, and Vercel AI Gateway is a fit for teams using the AI SDK; verify the exact behavior you need in each case.
Treat failover as application behavior, not a checkbox. Ask which errors trigger a fallback, whether the gateway retries automatically, and how it handles timeouts, rate limits and partial responses. Test these cases in staging with the models and providers you plan to use. Check whether routing rules, logs and cache behavior can be configured without changing application code. Keep a direct-provider path or a controlled rollback plan if gateway availability becomes a dependency.
05Work out cost, limits and ownership
A free plan or low starting price is not a production cost estimate. The listed starting prices vary: Eden AI starts at $0, Portkey at $49, NanoGPT at $12 and Helicone at $79; several other vendors do not publish a starting price. Ask how model usage is billed, whether the gateway adds charges, what free-tier limits apply, and what happens when a limit is reached. Estimate spend using your expected request volume, model mix and prompt sizes.
Decide who will operate the gateway. LiteLLM, Portkey and Helicone are listed as open source and self-hostable, which brings deployment, upgrades and credential security into your own stack. For hosted services, check data handling, retention, export options and service limits before sending production traffic. Confirm the licence and current project status for open-source choices, and avoid placing credentials in a gateway until your team has assessed its maintenance and supply-chain risks.
Questions people ask
What is an LLM gateway?
An LLM gateway gives an application a shared interface for calling models from multiple providers. Depending on the product, it can also handle routing, provider failover, request logs, cost tracking or caching. The exact controls vary, so verify each feature against the vendor’s docs.
Which LLM gateway is best for production?
There is no single production choice for every stack. OpenRouter is our best overall pick for provider choice and failover; Vercel AI Gateway fits teams using the AI SDK. For any gateway, validate failure handling, limits, cost behavior and data controls with your own workload before adopting it.
Are there free LLM gateways?
Several products list free plans, including OpenRouter, Requesty, Portkey, LiteLLM, Vercel AI Gateway, Cloudflare AI Gateway, NanoGPT and Helicone. Eden AI lists a $0 starting price. These labels do not establish production limits or total usage costs; ask vendors for the applicable terms.
Which LLM gateways can I self-host?
LiteLLM, Portkey and Helicone are listed as open source and self-hostable. Self-hosting gives your team deployment responsibility as well as control: assess the licence, maintenance status, release process, credential handling and upgrade work before making one a production dependency.