GoModel is a self-hosted AI gateway written in Go. It puts many LLM providers behind one endpoint that accepts both OpenAI-compatible and Anthropic-compatible requests, so the official OpenAI and Anthropic SDKs work unchanged once their base URL points at the gateway. It positions itself as an alternative to LiteLLM and Portkey, and publishes reproducible benchmarks.
Supported providers include OpenAI, Anthropic, xAI, Google Gemini, Vertex AI, Cohere, DeepSeek, Groq, Fireworks AI, OpenRouter, Z.ai, Alibaba Cloud Model Studio, MiniMax, Xiaomi MiMo, Azure OpenAI, Amazon Bedrock, Ollama, SGLang, vLLM, llm-d, and any OpenAI-compatible provider. ChatGPT (the Codex backend) and Claude subscriptions also work as backends.
Beyond routing, the gateway caches responses, tracks cost and usage, enforces budgets and rate limits, and fails over to backup providers. A hierarchy of user paths scopes keys, model access, budgets, usage, and audit logs. A web dashboard shows usage analytics, live requests, and a playground. Configuration comes from defaults, a YAML file, environment variables, or the dashboard.
GoModel Pro is a commercial build of the same gateway that adds prompt compression, intelligent routing (which classifies each request as easy or hard and picks the healthiest, cheapest provider in that tier), and OIDC single sign-on for the dashboard.
Features
- Dual-format API: accepts OpenAI-compatible and Anthropic-compatible requests, so official SDKs work unchanged
- Caching: exact and semantic response caching
- Cost tracking and budgets: per-request cost estimates, spending breakdowns, and hard spend limits per user, team, or key
- Rate limits: request, token, and concurrency caps per user path, provider, or model
- Virtual models: aliases with round-robin or cost-based load balancing behind stable model names
- Failover: reroutes to backup providers with retries and circuit breakers
- Session keeping: pins a client session to one provider key so provider prompt caches stay warm
- Access control: model allowlists per group, user, and key along the user-path tree
- MCP gateway: aggregates MCP servers behind one authenticated endpoint
- Guardrails and plugins: request and response policies, stream filters, header edits, and routing strategies through one plugin contract
- Workflows: versioned per-request policies that scope cache, budgets, audit logging, guardrails, and failover
- Observability: Prometheus metrics, OpenTelemetry traces, audit logs, and live request streaming in the dashboard
