Off Grid AI runs generative models directly on a phone or a Mac: text generation from GGUF models, image generation, vision question answering, speech transcription, tool calling, and document analysis, all executing on the device's own CPU, GPU, or NPU with no account or API key. It is a React Native application published for Android, iOS, and Apple Silicon Macs through Mac Catalyst.
Text generation covers Qwen 3, Llama 3.2, Gemma 3, Phi-4, and any GGUF file the user supplies, with streaming output, thinking mode, and Markdown rendering. Acceleration is detected automatically and overridable in settings: Adreno GPUs run through OpenCL, Apple Silicon through Metal, and the Snapdragon Hexagon NPU is marked experimental because it accelerates only Q40 and Q80 quantizations, silently falls back to CPU on a K-quant, and produces garbled output for some model architectures. Models that can use the GPU or NPU are badged in the model list so the quantization can be checked before a multi-gigabyte download.
| Task | Flagship | Mid-range |
|---|---|---|
| Text generation (CPU) | 15-30 tok/s | 5-15 tok/s |
| Text generation (GPU / OpenCL) | 20-40 tok/s | — |
| Image generation (NPU) | 5-10s | — |
| Image generation (CPU) | ~15s | ~30s |
| Vision inference | ~7s | ~15s |
| Voice transcription | real-time | real-time |
Those figures were measured on Snapdragon 8 Gen 2/3 and Apple A17 Pro and vary with model size and quantization.
Because RAM is finite, the model manager shows which models are resident and what each costs, with a per-model eject and three loading policies: Lean keeps one model in memory, Balanced co-resides models that fit and swaps those that do not, and Aggressive commits a larger share of RAM. A refused load can be overridden, and when a model is evicted mid-conversation the chat says so and offers to reload it.
A paid tier adds on-device Kokoro text-to-speech for spoken replies, custom personas with their own prompt, voice, and persistent memory, and connections to Calendar, email, and MCP servers such as Linear, Notion, and GitHub, where the assistant drafts an action and waits for approval before anything is sent. Phone and Mac sync over the local network rather than a relay.
Features
- On-device text generation: GGUF models including Qwen 3, Llama 3.2, Gemma 3, and Phi-4, plus user-supplied .gguf files, with streaming and thinking mode
- Hardware acceleration: OpenCL on Adreno GPUs, Metal on Apple Silicon, and an experimental Hexagon NPU path limited to Q40 and Q80 quantizations
- Image generation: on-device Stable Diffusion with real-time preview, NPU-accelerated on Snapdragon and Core ML on iOS, across 20+ models
- Vision: camera input answered by SmolVLM, Qwen3-VL, or Gemma 3n for documents, scenes, and receipts
- Speech to text: on-device Whisper transcription with hold-to-record, with no audio leaving the device
- Tool calling: built-in web search, calculator, date and time, device info, and knowledge base search, with an automatic tool loop and runaway prevention
- Project knowledge base: PDFs and text documents chunked, embedded on-device with a bundled MiniLM model, stored in SQLite, and retrieved by cosine similarity
- Remote servers: connection to any OpenAI-compatible server on the local network, with automatic model discovery, SSE streaming, and keys in the system keychain
- Document analysis: PDFs, code files, and CSVs attached to conversations, with native PDF text extraction on both platforms
- Prompt enhancement: the local text model expands a short prompt into a detailed image-generation prompt
- Memory controls: resident-model view with per-model eject and Lean, Balanced, or Aggressive loading policies
- Download manager: three concurrent downloads with a FIFO queue, pause, resume, retry, and cancel, surviving app backgrounding
