Apache APISIX is a dynamic API gateway built on NGINX and etcd. It handles north-south traffic at the edge and east-west traffic between services, and can also run as a Kubernetes ingress controller. Routes, upstreams, certificates, and plugins are hot-loaded from the etcd configuration store without restarts, which suits API management in microservice architectures.
Through its plugin system APISIX also acts as an AI gateway: it proxies requests to different LLM providers behind one interface, load-balances with retries and fallbacks across models, applies token-based rate limiting to control cost, and enforces authentication and traffic policies on AI workloads. The mcp-bridge plugin converts stdio-based MCP servers into HTTP SSE services that the gateway can scale.
You configure the gateway through a REST Admin API (or a standalone YAML mode for Kubernetes), and extend it with plugins written in Lua or through Java, Go, and Python plugin runners. Nodes are stateless and cluster through etcd, with routes matched by a radix tree that can use any NGINX variable, GraphQL attributes, IPv6, priorities, and TTLs as conditions.
Features
- AI proxying: routes LLM traffic to multiple providers through a unified interface with load balancing, retries, and fallbacks
- Token-based rate limiting: caps LLM usage by token count to protect upstream models and control spend
- MCP bridge: exposes stdio MCP servers as HTTP SSE services behind the gateway
- Protocols: HTTP(S), HTTP/3 with QUIC, WebSocket, gRPC and gRPC-Web, gRPC-to-HTTP/JSON transcoding, TCP/UDP, MQTT, and Dubbo proxying
- Traffic management: weighted round-robin and consistent-hash load balancing, health checks, circuit breaking, traffic split, proxy mirroring, canary and blue-green releases
- Request and response rewrite: modifies host, URI, method, headers, and response status or body in flight
- Authentication: key-auth, JWT, basic-auth, OpenID Connect, Keycloak, Casdoor, Casbin, and wolf-rbac plugins
- Access control: IP and referer allow/deny lists, CORS, CSRF protection, URI blocking, request validation, and built-in anti-ReDoS policies
- Rate limits: per-request, per-count, and per-concurrency limits, plus global rules applied to every request
- Observability: Prometheus metrics, Zipkin tracing, Apache SkyWalking, and fault injection
- Service discovery: etcd built in, plus Consul, Nacos, Eureka, and ZooKeeper
- Deployment: runs from bare metal to Kubernetes on x86 and ARM64, with a dashboard and CLI for operations
