Gateway

Portkey's AI Gateway is a proxy that sits between your application and language, vision, audio, and image model providers, exposing them through one OpenAI-compatible API. You run it locally with npx @portkey-ai/gateway (it listens on port 8787 with a console for local logs), deploy it on Docker, Node.js, Cloudflare Workers, or Replit, or use the hosted Portkey Cloud. Clients include the JS and Python SDKs, the OpenAI SDKs, REST, and framework integrations for LangChain, LlamaIndex, Autogen, and CrewAI.

Routing behaviour is expressed in configs attached to a client: retry counts, fallback chains that trigger on specified errors, weighted load balancing across keys or providers, conditional routing, request timeouts, and input or output guardrails that can deny a response (for example, blocking replies containing a given word). The gateway covers 45+ providers and 8+ agent frameworks, and a companion Portkey Models database supplies pricing for 2,300+ models across 40+ providers for cost tracking.

An MCP Gateway extends the same control plane to Model Context Protocol servers: a single authentication layer, per-team and per-user access control, logging of every tool call with parameters, response, and latency, and forwarding of user identity to the servers. It works with Claude Desktop, Cursor, VS Code, and other MCP clients. Semantic caching, provider cost optimization, prompt template management, role-based access control, and private deployments on AWS, Azure, GCP, OpenShift, and Kubernetes are part of the hosted and enterprise editions; a 2.0 pre-release branch is moving the enterprise gateway core into the open-source project.

Features

• Unified API: one OpenAI-style request signature for chat, vision, text-to-speech, speech-to-text, image generation, and OpenAI realtime APIs over websockets
• Fallbacks: switch to another provider or model on specified error types
• Automatic retries: up to 5 attempts with exponential backoff
• Load balancing: weighted distribution of requests across API keys or providers
• Request timeouts: per-request limits that terminate slow calls
• Guardrails: 40+ prebuilt input and output checks, plus bring-your-own and partner guardrails
• Key management: use your own provider keys or generate virtual keys
• MCP Gateway: authentication, access control, identity forwarding, and per-call observability for MCP servers
• Caching and analytics: response caching (semantic caching in hosted editions) and usage dashboards for volume, latency, cost, and error rates
• Agent framework support: Autogen, CrewAI, LangChain, LlamaIndex, Phidata, Control Flow, and custom agents
• Deployment options: local npx, Docker, Node.js, Cloudflare Workers, Replit, hosted cloud, or private cloud on AWS, Azure, GCP, OpenShift, and Kubernetes
• Model pricing data: the Portkey Models database backs cost tracking for 2,300+ models