ZINC (Zig INferenCe Engine) runs open-weight GGUF models on consumer GPUs. It ships as a single Zig binary, with no Python dependency, that provides a command line, a browser chat, a model manager, and an OpenAI-compatible API. Existing OpenAI clients and SDKs work against the API unchanged.
Each GPU backend has its own kernels and is benchmarked separately against llama.cpp on the same GGUF files, prompts, and warmups. On AMD, the ROCm backend processes prompts faster than the Vulkan backend, while generation speed is close between the two. Qwen 3.8 models can use their built-in NextN block for speculative decoding, which drafts tokens that the full model verifies in one batched pass, with identical output.
| Hardware | Backend | Status |
|---|---|---|
| AMD Radeon (RDNA3/RDNA4) | ROCm/HIP | Supported |
| AMD Radeon (RDNA3/RDNA4) | Vulkan | Supported |
| Intel Arc (Xe2/Battlemage) | Vulkan | Supported |
| Apple Silicon | Metal | Supported |
| NVIDIA RTX | CUDA | Experimental |
On a Radeon AI PRO R9700 with ROCm, ZINC was ahead of llama.cpp in prompt processing (1.07x to 1.9x on a short chat prompt) and token generation (1.02x to 1.28x) across six models, measured without speculation. Current tuning covers Qwen 3.5, Qwen 3.6, Qwen 3.8, Gemma 4, and Muse Glimmer, including mixture-of-experts and hybrid-attention architectures.
Features
- Single binary: one Zig program with a CLI, browser chat, model manager, and API
- OpenAI-compatible API: chat completions and model listing for existing clients and SDKs
- GPU backends: ROCm and Vulkan on AMD, Vulkan on Intel Arc, Metal on Apple Silicon, experimental CUDA on NVIDIA
- Model catalog: loads local GGUF files or pulls from a managed catalog and Hugging Face
- Speculative decoding: uses the NextN block in Qwen 3.8 to draft tokens verified in one batched pass
- Published benchmarks: per-backend comparisons with llama.cpp, including rows where llama.cpp is ahead
