MLX-Serve is a local inference server for Apple Silicon Macs, written in Zig with no Python at runtime. It runs MLX-format models natively and any GGUF model through an embedded llama.cpp, routing by format, and serves DeepSeek V4 Flash through the dedicated ds4 engine.
One server exposes four API surfaces: OpenAI chat completions and Responses (with a WebSocket transport), Anthropic Messages, and the Ollama API. Clients that speak any of these, such as Claude Code, the OpenAI SDK, Continue, Open WebUI, and existing Ollama-connected tools, point at it without changes. The serving surface covers streaming, tool calls with schema-driven repair, JSON-schema constrained decoding, logprobs, vision, and reasoning output.
Beyond text, the same server generates images, video, music, speech with voice cloning, and 3D models on MLX. MLX Core, a native macOS menu-bar app, wraps the server with chat, an agent mode with MCP tool calling, model management, and settings for every server flag.
Head-to-head comparison against other local engines:
| MLX-Serve | LM Studio | Ollama | mlx-lm | |
|---|---|---|---|---|
| MLX models | yes | yes | partial | yes |
| GGUF models | yes (embedded) | yes | yes | no |
| Anthropic Messages API | yes | partial | no | no |
| Ollama API | yes | no | native | no |
| Built-in agent loop and MCP client | yes | no | no | no |
| Decode speed vs LM Studio (geomean, identical weights) | +26% | baseline | about −15% (GGUF, estimated) | +11% |
Features
- Model formats: native MLX dispatch for Gemma, Qwen, Llama, Mistral, Nemotron-H, DeepSeek V4 Flash, and others, plus any GGUF through embedded llama.cpp
- Multiple API surfaces: OpenAI chat completions and Responses, Anthropic Messages, and the Ollama API on one endpoint
- Coding-agent launchers: one-click or CLI launch of Claude Code, Pi, OpenCode, Codex, Hermes, Aider, and others, preconfigured with the server's context window
- Speculative decoding: prompt lookup, model-shipped draft companions, the Gemma 4 drafter, and native Qwen multi-token prediction
- Serving performance: custom Metal kernels, continuous batching, 4-bit and 8-bit KV-cache quantization, and prefix and tokenize caches
- Media generation: image generation and editing, video, music, speech with voice cloning, and image-to-3D, over HTTP endpoints or in chat
- MLX Core app: menu-bar chat with 10 built-in tools and per-tool approvals, an MCP marketplace, skills, persistent memory, and folder RAG
- Agent Sandbox: runs every agent shell command in an isolated Linux VM
- LAN model sharing: one Mac uses another Mac's models over Bonjour
- Web console: browser UI for chat, media generation, a results library, live monitoring, and the API reference
- Ollama-style CLI: run, pull, list, and serve models by short name or Hugging Face id
