LiteRT is Google's runtime for running machine-learning and generative AI models on-device, and the successor to TensorFlow Lite. It takes models from PyTorch, TensorFlow, and JAX, converts and quantizes them into the .tflite or .litertlm format, and runs them on CPU, GPU, or NPU across mobile, desktop, web, and IoT targets.
Version 2 centers on the Compiled Model API, which selects an accelerator automatically, runs inference asynchronously, and handles I/O buffers without explicit delegates. A single API reaches NPUs from several chipset vendors, and GPU inference runs through the ML Drift backend. TensorFlow Lite's Interpreter API is in maintenance mode, and new code targets the Compiled Model API.
LiteRT is one part of the Google AI Edge tools: LiteRT Torch converts PyTorch models, the AI-Edge Quantizer optimizes converted models, LiteRT-LM runs large language models such as Gemma on edge devices, LiteRT.js runs models in the browser over WebGPU and WebAssembly, and MediaPipe builds cross-platform media pipelines on top. Converted models, including the Gemma 4 family and speech-recognition models, are published to a LiteRT community space on Hugging Face.
| Platform | GPU APIs | NPU / accelerators |
|---|---|---|
| Android | OpenCL, OpenGL | Broadcom, Google Tensor, Intel, MediaTek, Qualcomm, S.LSI |
| iOS | Metal | Apple Neural Engine (coming soon) |
| Linux | WebGPU | Broadcom, Intel |
| macOS | WebGPU, Metal | Apple Neural Engine (coming soon) |
| Windows | WebGPU | Intel |
| Web | WebGPU | WebNN (coming soon) |
| IoT | WebGPU | Raspberry Pi (coming soon) |
Features
- Compiled Model API: automatic accelerator selection, asynchronous execution, and efficient I/O buffer handling without manual delegates
- Unified NPU access: one API for NPUs from Google Tensor, Qualcomm, MediaTek, Intel, Broadcom, and S.LSI
- GPU acceleration: ML Drift GPU backend with buffer interoperability across GPU buffer types
- Model conversion: exports from PyTorch, TensorFlow, and JAX to .tflite, and LLMs to .litertlm
- On-device generative AI: runs quantized LLMs and diffusion models through LiteRT-LM
- Web inference: LiteRT.js runs models client-side in the browser over WebGPU and WebAssembly
- C++ Tensor API: a tensor-centric C++ library for graph authoring and memory control on mobile devices
- Agent skills: SKILL.md playbooks that guide coding agents through conversion, quantization, on-device verification, benchmarking, app scaffolding, and TensorFlow Lite migration
- LiteRT CLI: a command-line toolkit for model work from coding agents