Haystack

Haystack is a Python framework for building LLM applications out of modular pipelines and agent workflows. You decide explicitly how retrieval, routing, memory, and generation fit together, and use the same building blocks for retrieval-augmented generation (RAG), semantic search, question answering, multimodal applications, and autonomous agents.

A Pipeline connects components such as retrievers, rankers, tool callers, memory stores, generators, and evaluators. Pipelines support loops, branches, and conditional logic, so you control how context is retrieved, ranked, filtered, combined, and routed before it reaches the model. The same Pipeline runs synchronously or asynchronously and streams output token by token. Pipelines are serializable and cloud-agnostic, and run on Kubernetes.

The Agent component adds tool calling on top of pipelines. Lifecycle hooks such as before_llm, before_tool, and on_exit let you add guardrails and custom logic, and each run tracks step count, token usage, and tool calls. Agents can run tool calls concurrently, and SkillToolset loads skill descriptions into context only when they are needed. Agent Pack supplies ready-made agents, such as a deep research agent and an advanced RAG agent.

Haystack integrates with model providers including OpenAI, Mistral, Anthropic, Cohere, Hugging Face, Google, Azure OpenAI, AWS Bedrock, and local models, and with vector stores and search engines such as Weaviate, Pinecone, and Elasticsearch. Components can be swapped without rewriting the rest of the system. The companion Hayhooks project serves pipelines and agents as REST APIs, MCP servers, or OpenAI-compatible chat completion endpoints. A commercial Haystack Enterprise Platform, managed or self-hosted, adds visual pipeline design, testing workflows, access controls, and observability.