BaseRT is a local model runtime for Apple Silicon Macs, installed with a single shell command and run entirely on the device. You serve an open-weight model with a serve command, point a local coding agent at it, and no API key is needed and no request leaves the machine.
Published measurements on an Apple M5 Pro compare BaseRT with MLX and llama.cpp: up to 6.4× the prefill throughput of llama.cpp and up to 3.9× that of MLX, and up to 1.33× on decode. The compared workloads cover Qwen3 0.6B, Llama 3.2 1B and 3B, Qwen3 30B-A3B, and Gemma 4 E2B at 4-bit and 8-bit quantisation.
A plugin connects a local coding agent to the running server, so an on-device model backs the agent session. Technical reports, documentation, and a community channel are linked from the project's site.
Features
- One-command install: a single install script sets the runtime up
- On-device inference: models run locally, with no API keys and no data leaving the machine
- Model serving: a serve command hosts an open-weight model for local clients
- Coding-agent plugin: an installable plugin points a local agent at the running server
- Quantised open models: measured across Qwen3, Llama 3.2, and Gemma 4 checkpoints at 4-bit and 8-bit
- Published benchmarks: prefill and decode comparisons against MLX and llama.cpp on Apple M5 Pro
