Flama packages a trained model into a single portable .flm artifact and serves it over HTTP as an API. A model trained in scikit-learn, TensorFlow, PyTorch, or a large language model all take the same shape once packaged, so the serving path does not change with the framework a model came from, and flama get downloads and packages a model straight from the Hugging Face Hub.
For generative models, one served model answers on several wire protocols at once, so existing clients keep working against a local model without code changes:
| Dialect | Prefix | Representative routes |
|---|---|---|
| Native | (none) | /query/, /stream/, /chat/ |
| OpenAI | /openai | /v1/chat/completions, /v1/completions, /v1/responses, /v1/models |
| Anthropic | /anthropic | /v1/messages, /v1/models |
| Ollama | /ollama | /api/chat, /api/generate, /api/tags |
Every served model also carries a streaming chat page at /chat/ that renders Markdown, LaTeX, and Mermaid, and a model can be prompted straight from a terminal without a server. The same .flm file runs on vLLM on Linux with CUDA or on MLX on Apple Silicon, with the backend chosen at load time.
Model Context Protocol support is built in: you declare a tool, resource, or prompt with a decorator, mount an MCP server on the application, and the JSON Schema is derived from your type hints, with Tasks, Elicitation, and MCP Apps included.
Underneath is a full asynchronous API framework — SQLAlchemy-backed resources with CRUD methods, dependency injection through components, schemas via Pydantic, Typesystem, or Marshmallow, generated OpenAPI with Swagger UI and ReDoc, pagination, background tasks, JWT authentication, server-sent events and NDJSON streaming, and domain-driven-design patterns. Routing, JSON encoding, request parsing, and compression are compiled to native code and shipped as wheels for Python 3.10 to 3.14 on Linux, macOS, and Windows.
Features
- Portable model format: any mainstream framework's model becomes one .flm file
- Hub downloads: flama get fetches and packages a predictive or generative model in one command
- Multi-dialect serving: native, OpenAI, Anthropic, and Ollama endpoints side by side on one server
- Built-in chat UI: a streaming interface at /chat/ with Markdown, LaTeX, and Mermaid rendering
- Terminal inference: a stream subcommand pipes a prompt straight into a packaged model
- Native MCP: decorator-declared tools with schemas derived from type hints
- Serving backends: vLLM on Linux with CUDA, MLX on Apple Silicon
- Rust-powered core: routing, JSON encoding, request parsing, and compression compiled natively
- Resources and injection: CRUD resources over SQLAlchemy tables and component-based dependency injection
- Schema choice: Pydantic, Typesystem, or Marshmallow as optional extras
- API plumbing: generated OpenAPI, pagination, background tasks, lifespan events, and JWT authentication
- Version codemods: flama upgrade rewrites imports and renamed symbols across major versions
