Flama

Flama is a Python framework that turns machine learning and generative models into HTTP APIs. It packages a model from scikit-learn, TensorFlow, PyTorch, or an LLM into one portable .flm file, so every model looks the same to the API regardless of where it came from, and then serves that file without application code. A CLI command also downloads a model from the Hugging Face Hub and packages it directly.

A predictive model gets a metadata endpoint, a prediction endpoint, an OpenAPI schema, and Swagger UI. A generative model gets a native query and streaming API plus a built-in streaming chat UI that renders Markdown, LaTeX, and Mermaid. The same model can also expose OpenAI-, Anthropic-, and Ollama-compatible endpoints side by side, so existing clients for those APIs work against it unchanged. Generative models run on vLLM (Linux with CUDA) or MLX (Apple Silicon), and Flama picks the backend at load time.

Each .flm file is self-describing: it carries the framework and version, the model class, hyperparameters, training metrics, and auxiliary files, with the metadata ahead of the weights so inspection is a header read. A CI job can gate promotion on those metrics and score a reference set offline through the same code path the server uses. A single JSON spec file describes an application serving several models, and official container images serve it.

Flama also exposes tools, resources, and prompts to AI agents over the Model Context Protocol (MCP). A decorator declares each capability, and Flama derives the JSON Schema from type hints. Beyond model serving, it is a general API framework with a Rust core for routing, JSON encoding, request parsing, and compression, plus CRUD resources over SQLAlchemy, dependency injection, Pydantic, Typesystem, or Marshmallow schemas, pagination, JWT authentication, and Server-Sent Events.