Portkey AI Gateway
Fast open-source AI gateway that routes to 250+ LLMs through one API with load balancing, fallbacks, retries, and caching.
Portkey AI Gateway
Portkey's AI Gateway is a fast, open-source router that sits between your application and large language model providers, exposing 250+ models through a single, consistent API. It is built for production LLM orchestration, letting you switch or blend providers without rewriting code while keeping latency low with a tiny footprint.
Key features
- One unified API to OpenAI, Anthropic, Google, Azure, Bedrock, local models, and many more
- Automatic fallbacks and retries when a provider errors or times out
- Load balancing and weighted routing across keys and models
- Semantic and simple caching to cut cost and latency
- Request/response guardrails, timeouts, and observability hooks
The gateway is designed to add minimal overhead while giving teams resilience and control over model routing. Configuration is declarative, so you can define fallback chains, canary traffic splits, and per-request overrides via headers. It runs anywhere — as a local process, a container, or at the edge — and pairs with SDKs for common languages. With strong community traction, it has become a go-to open component for teams that need reliable, provider-agnostic LLM routing.
Curated mirror of the open-source Portkey AI Gateway (MIT). Get it from the source.
Compromise signals — malicious or tampered code (leaked secrets, backdoors, a dropped executable) — reduce the score, and known dependency CVEs carry a bounded penalty (they warrant review but never QUARANTINE — update the dependency to clear). Other dangerous-by-capability traits are risk surface, expected for some capabilities. Every finding is mapped to its OWASP control below.
Findings mapped to the OWASP Top 10 for LLM Applications (2025) and the OWASP Machine Learning Security Top 10. Expand any flagged control for the exact findings — compromise reduces the score; expected/risk-surface do not, except a known CVE, which carries a small bounded penalty (high/critical → Review).
The same gate an agent runs before installing (POST /api/v1/trust/portkey-ai-gateway/check). Click a policy:
Consume Portkey AI Gateway programmatically. Authenticate with an API key or session — see Authorize an agent.
# Agents: CHECK BEFORE YOU INSTALL (no auth) — score, grade, level, capability manifest
curl https://ai-supply.store/api/v1/trust/portkey-ai-gateway
# Gate against your org policy (returns { pass, violations })
curl -X POST https://ai-supply.store/api/v1/trust/portkey-ai-gateway/check \
-H "Content-Type: application/json" \
-d '{"minGrade":"B","denyPermissions":["shell"],"denyUnknownEgress":true}'
# CLI
npx ai-supply add portkey-ai-gateway
# REST (install → download)
curl -X POST https://ai-supply.store/api/v1/listings/portkey-ai-gateway/install \
-H "Authorization: Bearer $AIM_KEY"
# MCP tool
install_listing({ "slug": "portkey-ai-gateway" })OpenAPI spec →Curated mirror — latest upstream source. See the repository for tagged releases.