AIProxy is a Go-based AI gateway that sits between your applications and model providers. Clients send requests using the OpenAI Chat Completions, OpenAI Responses, Anthropic Messages, or Gemini protocol, and the gateway routes them across configured provider channels, converting between protocols where needed. A model that only speaks the Responses API can therefore be reached from a client that speaks Chat Completions, Claude Messages, or Gemini without client-side changes.
Routing selects channels by priority and observed error rate, retries with automatic error recovery, and load-balances across providers. Model mapping aliases model names to backends, and a built-in tokenizer removes the dependency on external tiktoken files. Text, image, audio, and document requests pass through, and prompt caching is tracked for billing.
Multi-tenant management isolates organizations from each other, issues tokens with optional subnet restrictions, applies requests-per-minute and tokens-per-minute limits and usage quotas per group, and supports per-group model pricing. A web management panel and Swagger API cover configuration and monitoring, with real-time alerts for balance, error-rate, and anomaly conditions, audit-level request and response logs, and analytics for volume, errors, RPM/TPM, and cost.
The gateway also hosts Model Context Protocol servers: public servers, private per-organization servers, embedded servers driven by configuration templates, and automatic conversion of OpenAPI specifications into MCP tools. Coding agents connect directly: Claude Code through ANTHROPICBASEURL, Gemini CLI through GOOGLEGEMINIBASEURL, and Codex through a modelproviders entry in its config. It deploys as a Docker container with SQLite by default, PostgreSQL optionally, and Redis for caching, and integrates with the Sealos platform and FastGPT.
Features
- Protocol conversion: two-way translation among OpenAI Chat Completions, OpenAI Responses, Claude Messages, and Gemini formats
- Channel routing: priority- and error-rate-based selection, retries, and load balancing across providers
- Multi-tenancy: organization isolation, token auth with subnet restrictions, per-group RPM/TPM quotas, and custom pricing
- Monitoring: alerts, audit logs, and analytics for volume, errors, RPM/TPM, and cost, plus channel performance views
- MCP hosting: public, organization, and embedded MCP servers, and OpenAPI-to-MCP tool generation
- Cache plugin: Redis or in-memory caching keyed by SHA256 of the request body, with TTL and size limits
- Web search plugin: Google, Bing, and Arxiv search with query rewriting, citation formatting, and user-controlled depth
- Think split plugin: extracts content from reasoning models into a reasoning_content field, streaming or not
- Stream fake plugin: runs non-streaming requests as internal streams to avoid timeouts, then reassembles the response
- Agent integrations: environment-variable or config-file setup for Claude Code, Gemini CLI, and Codex
- Deployment: Docker and Docker Compose, SQLite or PostgreSQL, optional Redis, and one-click deploy on Sealos
