Mlx Dspark

mlx-dspark runs two EAGLE-family speculative-decoding drafters natively on Apple Silicon through MLX: DeepSeek's DSpark (semi-autoregressive, from the DeepSpec codebase) and z-lab's DFlash (block diffusion). Both are lossless. The target model verifies every drafted token, so greedy output matches plain decoding, and sampling at a temperature draws an exact sample from the target. The targets are consumer-size models with published drafters (Gemma-4, Qwen3, Qwen3.6, Qwen3.8, LFM2.5, Muse-Glimmer, Ornith-1.0, Nemotron, MiniCPM5, and the ternary Bonsai-27B), not DeepSeek-V4 itself.

DSpark uses a five-layer parallel backbone that reads the target's hidden states and proposes a seven-token block at once, with a Markov head that corrects for the previous token and a confidence head that scores each position. DFlash denoises a whole 16-token block in one parallel pass and reuses the target's embeddings and output head. An auto mode picks the measured winner for each pair, the drafter loads from its Hugging Face checkpoint quantized to 4-bit, and the draft cap is derived from the machine's own measured verify and drafter cost curves rather than hard-coded.

The engine runs at the command line, from Python, and as a local single-user server that speaks three API dialects on one endpoint: OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages. Any OpenAI-compatible client that people otherwise point at llama.cpp or Ollama works by changing the base URL. The Anthropic dialect lets Claude Code run on a model on your own Mac through a launcher that scopes the configuration to that one process, the Responses dialect serves Codex, and Pi Coding Agent works against either API. Tool calling, multi-turn tool history, stop sequences, and system prompts are translated into each model family's own chat template and tool syntax, and a reasoning model's thinking output is lifted into thinking blocks.

Measured speedups over plain greedy decoding of the same model (M4 Pro, 48 GB, warm, 4-bit drafter, medians of 3 runs over chat, code, and math prompts):

target	speedup	chat / code / math	decode speed
Qwen3.8-27B (8-bit, DFlash 2 default)	3.79×	3.03× / 4.01× / 4.32×	31.1 tok/s
Gemma-4 12B (8-bit)	3.25×	2.62× / 2.90× / 4.24×	59.8 tok/s
MiniCPM5-2B (bf16)	3.16×	2.12× / 3.15× / 4.21×	159.7 tok/s
Ornith-1.0-9B (8-bit)	2.40×	2.21× / 2.53× / 2.48×	64.2 tok/s
Qwen3-8B (8-bit)	2.19×	1.71× / 2.03× / 2.83×	64.5 tok/s
Qwen3.6-35B-A3B (4-bit, MoE)	1.32×	1.05× / 1.24× / 1.67×	114.5 tok/s
Ternary-Bonsai-27B (2-bit)	1.07×	1.01× / 1.13× / 1.07×	27.2 tok/s

For coding agents, prefill dominates each turn because clients resend large system prompts (Claude Code around 18–26k tokens per request, Pi around 1.5k), so prefix-cache reuse moves wall clock more than acceptance length does. Prefix caching is on by default in every mode and reuses the previous turn's KV cache, trimming it for dense targets or snapshotting checkpoints for hybrid targets whose recurrent state cannot be rolled back. Continuous batching runs concurrent requests through one batched target forward for about 2.5× aggregate throughput, while a lone request takes the serial path.

A native Mac app wraps the same engine with saved chat sessions, a model manager that estimates whether a model fits the machine before download, live speculative-decoding telemetry, a roofline view of measured memory bandwidth against the plain-decode ceiling, a decoder race with a checked lossless verdict, and a menu-bar gauge of tok/s and model memory.