Haystack is a Python framework for orchestrating large language model (LLM) applications. You compose modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation, and use them to build retrieval-augmented generation (RAG) systems, multimodal applications, semantic search, question answering, and autonomous agents. Version 3.0 is the current major release.

The building blocks are components for retrieval, indexing, tool calling, memory, and evaluation, which you connect into a Pipeline that can include loops, branches, and conditional logic. The same pipeline runs synchronously or asynchronously and streams token by token, and an Agent can execute concurrent tool calls. Agents expose lifecycle hooks (beforellm, beforetool, onexit) for guardrails and custom logic, and report stepcount, token_usage, and tool calls for monitoring and cost control.

Haystack is model- and vendor-agnostic: integrations cover OpenAI, Mistral, Anthropic, Cohere, Hugging Face, Google, Azure OpenAI, AWS Bedrock, and local models. Related projects include Agent Pack (ready-made agents such as a deep research agent and an advanced RAG agent), SkillToolset for progressive skill discovery, and Hayhooks, which wraps pipelines and agents as REST APIs, MCP servers, or OpenAI-compatible chat endpoints. A commercial Haystack Enterprise platform adds observability, collaboration, governance, and access controls as a managed or self-hosted service.

Features

Pipelines: modular graphs of components with loops, branches, and conditional routing
Agents: tool-calling agents with lifecycle hooks and built-in tracking of steps, token usage, and tool calls
Async and streaming: one pipeline definition runs sync or async and streams tokens; agents run tool calls concurrently
Context engineering: explicit control over how information is retrieved, ranked, filtered, combined, and routed before reaching the model
Provider integrations: OpenAI, Mistral, Anthropic, Cohere, Hugging Face, Google, Azure OpenAI, AWS Bedrock, and local models
Agent Pack and skills: prebuilt agents plus SkillToolset, which loads skill descriptions into context only when needed
Custom components: a consistent interface for writing and sharing components through the haystack-core-integrations ecosystem
Serving: Hayhooks exposes pipelines and agents over HTTP, MCP, or OpenAI-compatible chat completion endpoints
Telemetry: anonymous component-usage statistics that you can opt out of