The Modular Platform is an AI inference stack that runs from GPU kernels up to a serving endpoint. Its two open components are the MAX framework, which serves models, and the Mojo language, in which the accelerator kernels are written.
MAX automatically optimizes kernels and request execution across accelerators, and the same model and codebase target NVIDIA, AMD, Trainium, TPU, Qualcomm, Intel, ARM, and Apple silicon. Its inference server exposes an OpenAI-compatible endpoint, so agents and applications call served models the same way they call a hosted API. Workloads run in Modular's hosted cloud or in your own VPC.
The open-source repository contains the Mojo compiler and standard library, the MAX accelerator kernel library, the MAX inference server, Python-based model pipelines, and code examples. Modular reports a 2x performance improvement over vLLM on diverse hardware.
Features
- MAX inference server: serves models behind an OpenAI-compatible endpoint
- Hardware portability: one codebase across NVIDIA, AMD, Trainium, TPU, Qualcomm, Intel, ARM, and Apple silicon
- Kernel library: MAX accelerator kernels written in Mojo, with automatic kernel and request optimization
- Model pipelines: Python-based graphs that define supported model architectures
- Mojo language: compiler and standard library for writing high-performance code
- Deployment options: Modular's hosted cloud or your own VPC
