Minions is a communication protocol that lets a small on-device language model collaborate with a frontier model in the cloud. The local model reads long contexts on the machine and exchanges only what it needs with the cloud model, which reduces cloud token cost while aiming to hold answer quality steady. The repository provides a working demonstration of the protocol alongside the research paper from Stanford's Hazy Research group.
Two protocol variants ship. Minion (singular) pairs one local client with one remote client over a small number of communication rounds. Minions (plural) splits work into structured, parallel subtasks, using a schema for the local model's output. Both are driven from a few lines of Python that instantiate a local client, a remote client, and the protocol object.
Local inference runs through one of three servers, and the cloud side connects to any of several hosted providers by API key.
| Component | Options |
|---|---|
| Local servers | ollama (CPU or non-NVIDIA), lemonade (AMD CPU/GPU/NPU), tokasaurus (NVIDIA, high-throughput) |
| Optional local backends | MLX-LM, Cartesia-MLX (Apple Silicon), llama-cpp-python |
| Cloud providers | OpenAI, Together AI, OpenRouter, Perplexity, DeepSeek, Anthropic, Mistral, MiniMax, Gemini, Groq, xAI, Cohere |
Features
- Local context reading: the on-device model processes long documents locally so only compact messages reach the cloud model
- Two protocols: Minion for single-round-trip collaboration and Minions for parallel, schema-structured subtasks
- Server choice: local inference through ollama, lemonade, or tokasaurus depending on available hardware
- Demo apps: a Streamlit application and a WebGPU app that runs the protocol entirely in the browser
- Secure Minions: an end-to-end encrypted local-remote protocol and chat system
- Inference estimator: command-line and Python tools for estimating inference cost and throughput
- Docker support: a container that bundles an Ollama service and a stdin/stdout JSON interface for running protocols
