The Modular Platform is a unified stack for AI inference, spanning custom GPU kernels through to a production serving endpoint. Its two headline pieces are the MAX framework, a hardware-agnostic serving framework, and Mojo, a systems language for writing the kernels underneath it.
MAX serves a model behind an OpenAI-compatible endpoint and automatically selects kernels and schedules request execution for the accelerator it finds. The same model and the same codebase are intended to run across NVIDIA, AMD, Trainium, TPU, Qualcomm, Intel, ARM and Apple silicon, and workloads can run in Modular's hosted cloud or in your own VPC. For agent builders, that makes it a place to host the model an agent calls rather than an agent framework in itself.
The open-source components live in one repository: the Mojo compiler and standard library, the MAX accelerator kernel library, the MAX inference server, and MAX model pipelines expressed as Python graphs, alongside code examples for both. Contributions are accepted to the standard library, the kernel library, model architectures, examples and docs, though not yet to the Mojo compiler itself. Repository code is Apache 2.0 with LLVM exceptions, while MAX usage falls under a separate community licence.
Features
- MAX inference server: serves models behind an OpenAI-compatible endpoint
- Automatic kernel selection: kernels and request execution are optimised for the accelerator in use, without per-target code
- Hardware portability: one codebase targeting NVIDIA, AMD, Trainium, TPU, Qualcomm, Intel, ARM and Apple silicon
- Mojo language: a systems language with its own compiler and standard library for authoring the accelerator kernels
- MAX kernel library: the accelerator library that MAX serving builds on
- Model pipelines: model architectures expressed as Python-based graphs
- Deployment choice: Modular's hosted cloud or your own VPC, delivered through a single container
