MicrosoftFoundry Local is an on-device AI runtime that an application ships with, so model inference runs inside the app's own process on the end user's machine. Prompts, audio, and responses stay on the device, the app keeps working offline, and there are no per-token costs, API keys, or cloud subscription involved.
It combines native SDKs for C#, JavaScript, Python, and Rust with a curated catalog of quantized models and a runtime (about 20 MB) built on ONNX Runtime. The SDK lifecycle is: create a manager, pick a model alias, download and cache it, load it, then call a chat or audio client. The runtime detects the device's hardware and picks an execution provider for NPU, GPU, or CPU.
The catalog covers chat models such as GPT OSS, Qwen, DeepSeek, Mistral, and Phi, plus Whisper for speech-to-text, and is deliberately limited to models tested on consumer hardware and small enough to distribute. An optional OpenAI-compatible server and a CLI exist for development workflows and for frameworks such as LangChain.
Foundry Local targets single-user, hardware-constrained devices on Windows, macOS (Apple silicon), and Linux. It does not provide request queuing, continuous batching, or GPU sharing for many concurrent clients, and points multi-user serving to runtimes such as vLLM. The runtime may send usage data to Microsoft.
Features
- Native SDKs: C#, JavaScript, Python, and Rust bindings run inference in-process without a separate server
- Curated model catalog: quantized chat models (GPT OSS, Qwen, DeepSeek, Mistral, Phi) and Whisper audio models
- Hardware selection: detects the device and picks an NPU, GPU, or CPU execution provider and model variant
- Model lifecycle: models download on first use, cache locally, and load the variant suited to the hardware
- Audio transcription: Whisper-based speech-to-text, file-based or streaming
- OpenAI-compatible API: accepts OpenAI request and response formats, including the Responses API format
- Optional local server: an OpenAI-compatible web server for multiple processes or frameworks such as LangChain
- CLI: lists and runs catalog models interactively during development
- Samples: chat, audio, tool calling, model management, an Electron app, and LangChain integrations across the four languages
