Lynkr is a self-hosted LLM gateway that sits between AI coding tools (Claude Code, Cursor, Codex, Cline, Continue) and the model providers behind them. It rewrites each request before it reaches the model: it strips tool schemas the request does not need, compresses large JSON tool results, answers near-duplicate prompts from a semantic cache, and routes the request to a model tier chosen by its difficulty. Clients connect by pointing their base URL at Lynkr, with no code changes.
Routing scores each request across four tiers (simple, medium, complex, reasoning) using an anchor-embedding classifier, a local LLM difficulty classifier, risk and agentic-pattern detectors, and forced-reasoning phrases. A conversation keeps one routing decision through a content fingerprint and escalates when the task outgrows its model. Outcomes train a kNN router and a bandit, and tier thresholds re-calibrate nightly from recorded traffic. A wrap mode applies the same routing to a Claude Code subscription session, sending simple tasks to local models.
Lynkr publishes benchmarks run against LiteLLM on the same backends (Ollama, Moonshot, Azure OpenAI):
| Mechanism | Lynkr | LiteLLM |
|---|---|---|
| JSON compression, 60-item grep result | 427 tokens | 3,458 tokens |
| Semantic cache, paraphrased repeat | 0 tokens, 171 ms | 2,857 tokens, 1,891 ms (cold) |
| Tier routing, 11 scenarios correct | 11/11 | 4/11 (heuristic router) |
The routing scenarios come from Lynkr's own regression suite.
Supported providers include Ollama, llama.cpp, LM Studio, OpenRouter, AWS Bedrock, Databricks, Azure OpenAI, Azure Anthropic, OpenAI, DeepSeek, Z.ai, Moonshot AI, Atlas Cloud, and Fireworks AI. Responses stream through every tier, with OpenAI-format upstreams converted into Anthropic events in flight.
Features
- Tool selection: classifies each request and drops irrelevant tool schemas before forwarding
- JSON compression: TOON and RTK compression of large tool results such as grep, file reads, and test output
- Semantic cache: serves near-identical prompts from cache using local embeddings, with no tokens billed on a hit
- Tier routing: sends simple requests to cheap or local models and escalates complex, risky, or autonomous tasks to stronger ones
- Learning router: a kNN router and bandit learn from outcomes, and thresholds re-calibrate nightly
- Wrap mode: applies routing and compression to Claude Code, Copilot, Cursor, Aider, and Codex sessions
- MCP Code Mode: replaces large sets of MCP tool schemas with four meta-tools (list, inspect, docs, execute)
- Streaming: passes Anthropic SSE through natively and converts OpenAI-format streams into Anthropic events
- Dashboard: spend and savings, tier mix, routing accuracy, request logs, provider health, and ranked overspend findings
- Claude Code status line: shows routed tier, served model, daily spend, and cache share after each turn
- Cost tracking: prices each request from a model-pricing registry and records it in telemetry
- Code intelligence: optional Graphify integration scores code complexity from the AST for routing
