OmniRoute is a self-hosted AI gateway, forked from 9Router, that places one OpenAI-compatible endpoint in front of several hundred model providers. Coding tools such as Claude Code, Codex, Cline, Cursor, and Copilot point at that endpoint, and OmniRoute chooses the upstream for each request, falls back when a provider fails or runs out of quota, and compresses the context on the way through. The gateway and its dashboard run on your own machine or on a remotely driven server, add no hosted prompt-processing hop, keep telemetry off by default, and encrypt credentials at rest.
Routing is organized around combos: a chain of models that OmniRoute moves along automatically when quota runs out, a provider fails, or costs spike. Setting the model to auto builds a virtual combo from the connected providers, scored live on 16 factors (health, quota, cost, latency, task fit, quality, session availability), with variants such as auto/coding, auto/fast, auto/cheap, and auto/chaos (parallel fan-out to a panel of models). Hand-built combos pick from 19 strategies, including priority, weighted, round-robin, cost-optimized, fusion (a model panel plus a judge), and pipeline (each target's output feeds the next). Fallback cascades across four provider tiers (subscription, API key, cheap, free) behind a per-provider circuit breaker, a per-connection cooldown, and a per-model lockout.
Every request passes through a compression pipeline of 12 composable engines, each toggleable per combo, with code blocks, URLs, and structured data preserved. The default stack runs the RTK engine (command-aware tool-result filtering) followed by the Caveman engine (rule-based prose compression), both built on ideas from the Rtk and Caveman projects. Presets combine the engines:
| Mode | Savings | Workload |
|---|---|---|
| Lite | ~15% | Always-on default |
| Standard (Caveman) | ~30% | Daily coding |
| Aggressive | ~50% | Long tool-heavy sessions |
| Ultra | ~75% | Maximum savings |
| RTK | 60–90% | Shell, test, build, and git output |
| Stacked (RTK then Caveman) | 78–95% | Mixed prompts and tool logs |
The provider catalog registers 357 providers across chat, media, search, local, cloud-agent, and system collections, 152 of them marked free, and the chat registry covers 229 providers and 2,554 provider-model pairs, including OpenAI, Anthropic, Gemini, xAI Grok, DeepSeek, Mistral, Qwen, Meta Llama, Groq, NVIDIA, MiniMax, Cohere, and Ollama as a first-class local provider. The REST surface mirrors the OpenAI API for chat, embeddings, images, audio, and OCR, and a models.dev sync feeds model specs and pricing into cost tracking.
Beyond the server, OmniRoute is a CLI with 80+ commands, including a TUI chat client and one-command launchers for seven coding CLIs (Claude Code, Codex, Aider, Goose, OpenCode, Qwen Code, and Gemini CLI). An agent can drive the gateway over MCP (110 tools under 33 scopes with an audit trail), A2A, the REST API, webhooks, or a remote CLI with scoped tokens. The codebase is TypeScript with a Next.js dashboard and SQLite storage.
Features
- Combos and auto-routing: a chain of models that OmniRoute advances automatically on quota exhaustion, failure, or cost spikes, with auto building a virtual combo scored live on 16 factors and variants for coding, speed, cost, offline headroom, and parallel fan-out
- 19 routing strategies: priority, fill-first, weighted, round-robin, power-of-two-choices, least-used, cost-optimized, headroom, reset-aware, context-relay, cache-optimized, last-known-good path, fusion (model panel plus judge), and pipeline chaining
- Tiered fallback and resilience: a four-tier cascade (subscription, API key, cheap, free) with a per-provider circuit breaker, per-connection cooldown with backoff, and per-model lockout
- 12-engine compression pipeline: Session-Dedup, CCR, Lite, RTK, Headroom, Relevance, Caveman, Aggressive, LLMLingua-2, Ultra, and OmniGlyph engines, toggleable per combo, with code blocks, URLs, and JSON preserved, and presets from ~15% to 78–95% savings
- Output styles and adaptive budget: cache-safe response-shaping instructions (terse prose, less code, action-first), a context-budget dial that escalates engines only to fit the model's window, and per-request or per-combo compression plans echoed in a response header
- Provider catalog: 357 registered providers, 152 marked free, covering OpenAI, Anthropic, Gemini, xAI, DeepSeek, Mistral, Qwen, Meta Llama, Groq, NVIDIA, MiniMax, Cohere, and local Ollama, with model specs and pricing synced from models.dev
- Coding-tool integrations: compatible with Claude Code, Codex, Cline, Kilo Code, Continue, Aider, OpenCode, Copilot CLI, Cursor, Hermes Agent, OpenClaw, Goose, and any OpenAI-compatible tool, plus one-command launchers and configuration writers for the common CLIs
- Agent control surface: MCP over stdio, HTTP, or SSE (110 tools, 33 scopes, audit trail), A2A with JSON-RPC 2.0 and SSE, webhooks to Slack, Discord, Telegram, or any URL, and a remote CLI with scoped tokens
- OpenAI-compatible REST API: chat, embeddings, images, audio, OCR, Batch and Files APIs, a WebSocket bridge, and cost and routing-decision headers on every response
- Memory and cache: opt-in memory with full-text search, int8-quantized vector embeddings, and typed decay, plus a semantic response cache
- Guardrails and privacy: a prompt-injection guard on every route, opt-in credential masking in both directions, API-key scoping, IP filtering, credentials encrypted at rest, and telemetry off by default
- Delivery targets: npm package, multi-arch Docker image, Electron desktop app, macOS menu-bar tray, Android via Termux, PWA, OpenCode provider plugin, and the OmniCopilot VS Code extension for Copilot Chat
