Higress is a cloud-native API gateway built on Istio and Envoy, extended with WebAssembly plugins written in Go, Rust, or JavaScript. It ships dozens of general-purpose plugins and a web console, and can run either inside Kubernetes or from a single Docker container.
Its AI gateway role puts one protocol in front of many model providers, so backend model routing can change without touching callers. That layer adds AI observability, multi-model load balancing with model-level fallback, token-based rate limiting, and response caching. Higress is also listed as a conformant Gateway API Inference Extension implementation, which supports inference-aware routing on Kubernetes.
The same plugin mechanism hosts Model Context Protocol (MCP) servers, giving agents a managed path to tools and services with unified authentication and authorisation, fine-grained rate limits, audit logs for every tool call, and dynamic updates that do not drop connections. A companion openapi-to-mcp tool converts OpenAPI specifications into remote MCP servers, and Higress manages LLM APIs and MCP APIs together.
Outside AI traffic, Higress works as a Kubernetes ingress controller — compatible with many NGINX Ingress Controller annotations and conformant with the Gateway API — as a microservice gateway that discovers services from Nacos, ZooKeeper, Consul, or Eureka and integrates with Dubbo and Sentinel, and as a security gateway with WAF plus key-auth, hmac-auth, jwt-auth, basic-auth, and OIDC strategies.
Request and response bodies are processed as true streams, including Server-Sent Events, so a Wasm plugin can handle streaming protocols without buffering whole payloads. Configuration changes apply in milliseconds rather than through a reload, which keeps long-lived connections intact.
Features
- Wasm plugin model: plugins in Go, Rust, or JavaScript run in sandbox isolation, upgrade independently, and hot-update without dropping traffic
- Unified LLM access: one protocol in front of mainstream model providers, with model-level fallback
- Token governance: token-based rate limiting, usage tracking, and response caching for repeated prompts
- MCP server hosting: MCP servers run as plugins with shared auth, rate limits, audit logs, and observability
- OpenAPI conversion: the openapi-to-mcp tool turns an OpenAPI specification into a remote MCP server
- Ingress controller: Gateway API and Ingress API support, compatible with many NGINX ingress annotations
- Service discovery: Nacos, ZooKeeper, Consul, and Eureka registries, with Dubbo and Sentinel integration
- Security: WAF, IP and cookie CC protection, and key-auth, hmac-auth, jwt-auth, basic-auth, and OIDC
- Streaming: full streaming of request and response bodies, including SSE, to limit memory use on AI traffic
- Deployment: one Docker command for a local all-in-one image, or Helm on Kubernetes with configurable image registries
- Certificates: Let's Encrypt integration issues and renews certificates automatically
- Console: a built-in UI on port 8001, with HTTP on 8080 and HTTPS on 8443
