MLC LLM is a machine learning compiler and deployment engine for large language models. Its goal is to let you develop, optimize, and deploy models natively on your own hardware rather than through a single vendor runtime.
Models are compiled and run on MLCEngine, one inference engine shared across every supported platform. MLCEngine exposes an OpenAI-compatible API through a REST server and through Python, JavaScript, iOS, and Android bindings, all backed by the same engine and compiler. Browser deployment is handled by the related WebLLM project. The compilation stack builds on TVM, TensorIR, and MetaSchedule.
Platform and GPU coverage:
| Platform | AMD GPU | NVIDIA GPU | Apple GPU | Intel GPU |
|---|---|---|---|---|
| Linux / Windows | Vulkan, ROCm | Vulkan, CUDA | N/A | Vulkan |
| macOS | Metal (dGPU) | N/A | Metal | Metal (iGPU) |
| Web browser | WebGPU and WASM (all) | |||
| iOS / iPadOS | Metal on Apple A-series GPU (all) | |||
| Android | OpenCL on Adreno GPU | OpenCL on Mali GPU |
Features
- ML compilation: compiles LLMs to native code for each target backend using the TVM stack
- Unified engine: MLCEngine serves every platform from one compiler and runtime
- OpenAI-compatible API: available via a REST server, Python, JavaScript, iOS, and Android
- GPU backends: CUDA, ROCm, Vulkan, Metal, OpenCL, and WebGPU
- Mobile and web targets: iOS, iPadOS, Android, and browsers through WebLLM
- Cross-vendor hardware: AMD, NVIDIA, Apple, and Intel GPUs on desktop, plus Adreno and Mali on Android