Flyte 2 is a Python orchestration runtime for ML pipelines, model serving, and AI agents. Workflows are plain Python: a task environment declares the container image, resources, secrets, and retry policy, and tasks are ordinary sync or async functions that branch, loop, and fan out with normal control flow rather than a static DAG or a YAML definition.
Durability is the central idea. The runtime records the inputs and outputs of every step, so a failed run resumes from the failed task with the same inputs instead of restarting the pipeline. It is infrastructure-aware: out-of-memory kills, preempted spot nodes, and lost GPUs trigger a resubmission with corrected resources, and transient errors such as API rate limits are retried with backoff while completed work is kept. Input and output between tasks is type-checked across native types, files, DataFrames, and dataclasses.
The same API serves long-running apps next to workflows: real-time model endpoints (including OpenAI-compatible LLM endpoints through vLLM, SGLang, or Ollama), FastAPI services, and Streamlit dashboards, with autoscaling and scale to zero. For agents, tools can be wrapped as tasks with every LLM call traced, and agent code can run in sandboxes. Integrations cover Spark, Ray, PyTorch Elastic, BigQuery, Snowflake, Weights & Biases, the OpenAI Agents SDK, LangGraph, and CrewAI.
Code runs locally, in a local devbox, or on a cluster, with a terminal UI for local development. The open-source Kubernetes-native backend for Flyte 2 is still in progress, and a hosted backend is available from Union.ai. Flyte 1 continues to be maintained separately, and Flyte is a graduated LF AI & Data Foundation project.
Features
- Pure Python workflows: tasks are plain functions in a task environment, with no DSL or YAML
- Logical durability: resumes from the failed task with the exact same inputs instead of rerunning the pipeline
- Compute durability: recovers from OOM kills, spot preemption, and lost GPUs by resubmitting with corrected resources
- Dynamic execution: branch, loop, and decide at runtime for agents and dynamic pipelines
- Parallelism: thousands of parallel tasks with map, fan-out, and streaming map-reduce
- Typed I/O: type-checked data passing with files, DataFrames, and dataclasses
- Model and app serving: OpenAI-compatible LLM endpoints via vLLM, SGLang, or Ollama, plus FastAPI apps and Streamlit dashboards with scale to zero
- Agent support: tools wrapped as tasks, traced LLM calls, and sandboxed agent code
- Ecosystem integrations: Spark, Ray, PyTorch Elastic, BigQuery, Snowflake, Weights & Biases, OpenAI Agents SDK, LangGraph, and CrewAI
- Run anywhere: the same code runs locally, in a devbox, or on a cluster
