Helicone is an AI gateway and LLM observability platform. Pointing an OpenAI-compatible client at ai-gateway.helicone.ai with a Helicone API key gives access to 100+ models from many providers through one key, with routing and automatic fallbacks, and logs every request for inspection. The platform also records traces and sessions for agents, chatbots, and document pipelines, and tracks cost, latency, and quality metrics.
Prompts can be versioned from production data and deployed through the gateway without code changes. A playground supports iterating on prompts, sessions, and traces in the UI, and datasets feed fine-tuning through partners OpenPipe and Autonomi. Metrics export to PostHog for custom dashboards. The hosted service is SOC 2 and GDPR compliant.
Helicone can be self-hosted with Docker Compose, or with a Helm chart for enterprise deployments. It consists of five services: a Next.js web frontend, a Cloudflare Workers proxy for logging, a Jawn log-collection server (Express and Tsoa), Supabase for the application database and auth, ClickHouse for analytics, and MinIO for log object storage.
Features
- AI gateway: one API key for 100+ models over the OpenAI API with routing and fallbacks
- Request logging: a one-line base-URL change logs OpenAI, Anthropic, Gemini, LangChain, Vercel AI SDK, and other calls
- Traces and sessions: inspect and debug multi-step agent runs
- Cost and latency analytics: per-request metrics with PostHog export
- Prompt management: versioned prompts deployed through the gateway
- Playground: test prompts, sessions, and traces in the UI
- Fine-tuning: datasets handed to OpenPipe or Autonomi
- Async logging: OpenLLMetry-based logging for JS/TS and Python
- Provider integrations: OpenAI, Azure OpenAI, Anthropic, Ollama, AWS Bedrock, Gemini API, Vertex AI, Anyscale, Together AI, Hyperbolic, Groq, DeepInfra, and Fireworks AI
- Framework integrations: LangChain, LlamaIndex, LangGraph, Vercel AI SDK, Semantic Kernel, CrewAI, ModelFusion, RAGAS, Open WebUI, MetaGPT, Open Devin, Mem0 EmbedChain, and Dify
- Self-hosting: Docker Compose or Helm deployments
