Catalogue
Submit a toolHarnesses, frameworks, tools, apps, and platforms for agent builders, each scored on real GitHub credibility and each with a credibility-gated forum.
35 tools · showing 1–24
ToolvLLM
Inference
vllm-project
LLM inference and serving library using PagedAttention and continuous batching, with an OpenAI-compatible API server
pagedattentionopenai-compatiblequantization
ToolAxonHub
CodingInterface
looplj
AI gateway that translates any SDK's requests to any provider, with tracing, RBAC, load balancing, and cost tracking
gatewayobservabilityauthtoken-optimization
ToolLiteLLM
Inference
BerriAI
Open-source AI gateway exposing 100+ LLM providers through one OpenAI-compatible interface, as a Python SDK or self-hosted proxy
openai-compatibleproxymulti-providergateway
ToolLocalAI
InferenceGenerative Media
mudler
Self-hosted engine that runs LLM, vision, voice, image, and video models on any hardware behind OpenAI-compatible APIs
local-firstopenai-compatiblellama-cppmultimodal
ToolOpen WebUI
Interface
open-webui
Self-hosted, offline-capable AI platform for Ollama and OpenAI-compatible models with RAG, tools, and multi-user access control
ollamaraglocal-firstopenai-compatible
ToolFreeLLMAPI
DeploymentCoding
tashfeenahmed
OpenAI-compatible proxy that stacks the free tiers of 16 LLM providers (~1.7B tokens/month) behind one /v1 endpoint — plus any custom OpenAI
gatewayopenai-compatiblefailoversmart-routing
Tool9Router
Coding
decolua
Unlimited AI coding. Connect Claude Code, Codex, Cursor, Cline, Copilot, Antigravity to free Claude/GPT/Gemini via 40+ providers.
gatewaymulti-providerfallback-routingtoken-optimization
ToolManifest
Inference
mnfst
Open-source LLM router exposing one OpenAI-compatible endpoint across API keys, subscriptions, and local models, with cost tracking
gatewaytoken-optimizationopenai-compatiblebyok
ToolJan
InterfaceInference
menloresearch
Desktop app for running open-weight LLMs locally or connecting to cloud providers, exposing an OpenAI-compatible local API
local-firstopenai-compatiblellama-cppdesktop
ToolSmg
Coding
lightseekorg
Engine-agnostic gateway that routes LLM requests across self-hosted and cloud backends with cache-aware load balancing
gatewayvllmopenai-compatible
ToolMesh LLM
Deployment
Mesh-LLM
Distributed LLM inference that pools GPUs across machines and serves one OpenAI-compatible API, splitting large models across nodes
distributed-inferenceopenai-compatiblegpulocal-first
ToolMlx Serve
InferenceGenerative MediaVoice
ddalcu
Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Includes MLX Core macOS app with chat, agent mo
mlxggufapple-siliconopenai-compatible
ToolSwitchyard
Inference
NVIDIA-NeMo
Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility -
gatewaylitellmopenai-compatibletoken-optimization
ToolApfel
InterfaceInference
Arthur-Ficial
The free AI already on your Mac. CLI tool, OpenAI-compatible server, and interactive chat — all on-device via Apple Intelligence. No API key
apple-intelligenceon-devicemacosopenai-compatible
PlatformExo
InferenceDeployment
exo-explore
Open-source distributed inference runtime that pools everyday devices into one local cluster to serve frontier models behind an OpenAI-compatible API.
distributed-inferencedevice-clusteropenai-compatiblelocal-first
Toolmistral.rs
Coding
EricLBuehler
Rust LLM inference engine with multimodal support, automatic quantization, and OpenAI- and Anthropic-compatible serving
quantizationopenai-compatiblemultimodallora
ToolWebLLM
Inference
mlc-ai
In-browser LLM inference engine running open models client-side on WebGPU with an OpenAI-compatible API
webgpubrowseropenai-compatiblejson-mode
ToolFlama
InferenceInterfaceConnectors
vortico
Ollama is fine for trying a model. vLLM becomes interesting when that model has to serve traffic: many users, agent work
openai-compatiblechatbotasgi
ToolFuXi
CodingInterface
fuxicodex
FuXi is a fast, self-contained AI coding agent that lives in your terminal — edit code, run commands, and drive tools, with cost-aware routi
productivityclicoding-agentopenai-compatible
ToolGoModel
InferenceMonitoring
ENTERPILOT
AI gateway written in Go. Lightweight unified OpenAI-compatible API for OpenAI, Anthropic, Gemini, Groq, xAI & Ollama. LiteLLM alternative w
gatewayproxyopenai-compatibletoken-optimization
ToolAIProxy
CodingMonitoring
labring
AI gateway with OpenAI-, Anthropic-, and Gemini-compatible routing, multi-tenant management, monitoring, and a plugin system
gatewaymulti-tenantopenai-compatiblemonitoring-dashboard
ToolMuna
InferenceDeployment
muna-ai
Compiles Python AI functions into self-contained native binaries and serves open models via an OpenAI-compatible client across cloud, edge and device.
openai-compatibleon-devicegpumodel-compilation
ToolGateway
Coding
Portkey-AI
Open-source AI gateway routing to 1,600+ LLMs through one API with retries, fallbacks, and guardrails
gatewayopenai-compatibleguardrailsfallback
ToolMLC LLM
Inference
mlc-ai
Machine-learning compiler and engine for deploying LLMs across AMD, NVIDIA, Apple, and Intel GPUs, browsers, iOS, and Android
cross-platformwebgpuon-devicequantization