Rapid-MLX is a local LLM inference server for Apple Silicon Macs, built on Apple's MLX framework and aimed at coding agents that depend on reliable tool calling. It exposes OpenAI-compatible endpoints (chat completions, responses, embeddings, images, audio, and videos) and the Anthropic Messages API, so clients such as Codex CLI and Claude Code connect to it without an adapter. It runs as a CLI server or through the Rapid-MLX Desktop macOS app.
The engine runs MLX kernels only, with continuous batching, a radix prefix cache that is saved to disk on shutdown and restored at startup, and a quantized KV cache. It ships 27 tool-call parser modules, including an auto-detect fallback. On a 48 GB M4 Pro Mac mini serving Qwen3.5-9B at 4-bit, decode measured 1.50× the speed of mlx-lm at the median of 18 streaming tasks and up to 4.27× on a whole-file code edit, a gain attributed to speculative decoding.
Twelve agent CLIs and three Python frameworks are tested against real weights on every release. Five Tier-1 agents (Claude Code, Codex CLI, Hermes, Aider, and DeepSeek Harness) must each complete a multi-step bug-fix task against a local 35B model before a version can publish. Beyond text, the server generates images and video (Wan, CogVideoX-Fun, and LTX backends), handles speech, transcription, and voice cloning, and serves typed decision models through a System One API.
A built-in recommendation catalog picks a model by the Mac's RAM:
| RAM | Recommended model | Peak RSS |
|---|---|---|
| 8–15 GB | lfm2.5-2.6b-4bit | 3.0 GB |
| 16–17 GB | qwen3.5-4b-4bit | 6.0 GB |
| 18–23 GB | qwen3.5-9b-4bit | 8.7 GB |
| 24–31 GB | bonsai-27b-2bit | 13.0 GB |
| 32 GB+ | qwen3.8-27b-4bit | 20.0 GB |
Rapid-MLX began as vLLM-MLX and has since been renamed and largely rewritten. It builds on MLX, mlx-lm, mlx-vlm, and mlx-audio.
Features
- OpenAI and Anthropic APIs: serves chat completions, responses, and Anthropic Messages endpoints so agent clients connect without an adapter
- Tool-call parsing: 27 parser modules plus an auto-detect fallback turn model output into structured tool calls
- Caching and batching: continuous batching, a radix prefix cache persisted across restarts, and a quantized KV cache
- Speculative decoding: MTP decoding on supported models, measured at up to 4.27× mlx-lm decode speed on a whole-file code edit
- Tested agent support: 12 agent CLIs and 3 frameworks (LangChain, Pydantic AI, smolagents) verified against real weights each release, with 5 Tier-1 agents gating publication
- Agent auto-setup: a single command configures supported agents such as Codex CLI to use the local server
- Model catalog: 270 aliases across text, image, video, and audio models, with RAM-tier recommendations per Mac
- Image and video generation: OpenAI-compatible Images and Videos APIs backed by FLUX, Wan, CogVideoX-Fun, and LTX models
- Audio: text-to-speech, transcription (Whisper, Parakeet, SenseVoice), and voice cloning behind OpenAI-compatible audio endpoints
- Typed decisions: a System One API returns choice and score decisions from compact classifier models
- Desktop app: a macOS app with model picker and an Images tab alongside the CLI server
- Benchmarking: a reproducible per-Mac benchmark that saves results privately, with consent-gated sharing to community results
