Helicone is an AI gateway and LLM observability platform. Pointing an OpenAI-compatible client at ai-gateway.helicone.ai with a Helicone API key gives access to 100+ models from many providers through one key, with routing and automatic fallbacks, and logs every request for inspection. The platform also records traces and sessions for agents, chatbots, and document pipelines, and tracks cost, latency, and quality metrics.

Prompts can be versioned from production data and deployed through the gateway without code changes. A playground supports iterating on prompts, sessions, and traces in the UI, and datasets feed fine-tuning through partners OpenPipe and Autonomi. Metrics export to PostHog for custom dashboards. The hosted service is SOC 2 and GDPR compliant.

Helicone can be self-hosted with Docker Compose, or with a Helm chart for enterprise deployments. It consists of five services: a Next.js web frontend, a Cloudflare Workers proxy for logging, a Jawn log-collection server (Express and Tsoa), Supabase for the application database and auth, ClickHouse for analytics, and MinIO for log object storage.

Features

AI gateway: one API key for 100+ models over the OpenAI API with routing and fallbacks
Request logging: a one-line base-URL change logs OpenAI, Anthropic, Gemini, LangChain, Vercel AI SDK, and other calls
Traces and sessions: inspect and debug multi-step agent runs
Cost and latency analytics: per-request metrics with PostHog export
Prompt management: versioned prompts deployed through the gateway
Playground: test prompts, sessions, and traces in the UI
Fine-tuning: datasets handed to OpenPipe or Autonomi
Async logging: OpenLLMetry-based logging for JS/TS and Python
Provider integrations: OpenAI, Azure OpenAI, Anthropic, Ollama, AWS Bedrock, Gemini API, Vertex AI, Anyscale, Together AI, Hyperbolic, Groq, DeepInfra, and Fireworks AI
Framework integrations: LangChain, LlamaIndex, LangGraph, Vercel AI SDK, Semantic Kernel, CrewAI, ModelFusion, RAGAS, Open WebUI, MetaGPT, Open Devin, Mem0 EmbedChain, and Dify
Self-hosting: Docker Compose or Helm deployments

Discussing Helicone· 1

1 thread · General

Mintlify acquired Helicone, in maintenance mode