One API puts many large-language-model providers behind one OpenAI-format API. You register each upstream provider as a channel, issue access tokens to users, and point any OpenAI-compatible client at the deployment by changing its API base URL and key.
Requests are relayed unchanged to channels that already speak the OpenAI format and rewritten for providers that do not. Several channels serving the same model are used in a load-balanced pool, with automatic retry after a failure; a token can pin one channel by appending its ID to the key. Channels can be tested and disabled automatically when their success rate falls below a configurable threshold.
The system is built around redistributing access rather than just proxying it. Quota is tracked per user and per token, users and channels are grouped with their own price multipliers, and redemption codes top up balances. A token carries its own expiry, quota, permitted IP ranges, and allowed model list. Quota for a request is calculated as group multiplier x model multiplier x (prompt tokens + completion tokens x completion multiplier).
Deployment is a single executable with a prepared Docker image. SQLite is the default store; MySQL or PostgreSQL is selected with SQL_DSN, and Redis can be added as a cache. Multi-node setups share one database and a session secret, designate master and slave nodes, and periodically resynchronise configuration. Sign-in supports email, GitHub, Feishu, and WeChat, with Cloudflare Turnstile verification available.
Features
- Provider coverage: OpenAI and Azure OpenAI, Anthropic Claude (including AWS), Google PaLM2 and Gemini, Mistral, DeepSeek, xAI, Cohere, Groq, Ollama, Doubao, ERNIE, Qwen, Spark, ChatGLM, 360, Hunyuan, Moonshot, Baichuan, MiniMax, Cloudflare Workers AI, together.ai, novita.ai, SiliconCloud, and DeepL
- Load balancing: multiple channels for one model, with automatic retry and optional channel disabling by success rate
- Token management: expiry, quota, permitted IP ranges, and allowed models per token
- Quota accounting: per-user and per-token balances, itemised usage records, and optional display in US dollars
- Groups and multipliers: user groups and channel groups with independent price multipliers
- Redemption codes: batch generation and export of codes that credit an account
- Model mapping: redirect a requested model name to another on the way to the channel
- Streaming and images: streamed responses and image-generation endpoints alongside chat completions
- Management API: system access tokens drive the admin API so behaviour can be extended without forking
- Multi-node deployment: shared database, master and slave node types, Redis caching, and periodic config sync
- Sign-in options: email with an optional allowlist, GitHub, Feishu, and WeChat, plus Cloudflare Turnstile
- Customisation: system name, logo, footer, announcement, home and about pages, and swappable themes