freshcrate
Skin:/
Home > Frameworks > haystack

haystack

Open-source AI orchestration framework for building context-engineered, production-ready LLM applications. Design modular pipelines and agent workflows with explicit control over retrieval, routing, m

Why this rank:Strong adoptionRecent releaseHealthy release cadence

Description

Open-source AI orchestration framework for building context-engineered, production-ready LLM applications. Design modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation. Built for scalable agents, RAG, multimodal applications, semantic search, and conversational systems.

README

Blue banner with the Haystack logo and the text โ€˜haystack by deepset โ€“ The Open Source AI Framework for Production Ready RAG & Agentsโ€™ surrounded by abstract icons representing search, documents, agents, pipelines, and cloud systems.
CI/CD Tests types - Mypy Coverage Status Ruff
Docs Website
Package PyPI PyPI - Downloads PyPI - Python Version Conda Version GitHub License Compliance
Meta Discord Twitter Follow

Haystack is an open-source AI orchestration framework for building production-ready LLM applications in Python.

Design modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation. Build scalable RAG systems, multimodal applications, semantic search, question answering, and autonomous agents, all in a transparent architecture that lets you experiment, customize deeply, and deploy with confidence.

Table of Contents

Installation

The simplest way to get Haystack is via pip:

pip install haystack-ai

Install nightly pre-releases to try the newest features:

pip install --pre haystack-ai

Haystack supports multiple installation methods, including Docker images. For a comprehensive guide, please refer to the documentation.

Documentation

If you're new to the project, check out "What is Haystack?" then go through the "Get Started Guide" and build your first LLM application in a matter of minutes. Keep learning with the tutorials. For more advanced use cases, or just to get some inspiration, you can browse our Haystack recipes in the Cookbook.

At any given point, hit the documentation to learn more about Haystack, what it can do for you, and the technology behind.

Features

Built for context engineering
Design flexible systems with explicit control over how information is retrieved, ranked, filtered, combined, structured, and routed before it reaches the model. Define pipelines and agent workflows where retrieval, memory, tools, and generation are transparent and traceable.

Model- and vendor-agnostic
Integrate with OpenAI, Mistral, Anthropic, Cohere, Hugging Face, Azure OpenAI, AWS Bedrock, local models, and many others. Swap models or infrastructure components without rewriting your system.

Modular and customizable
Use built-in components for retrieval, indexing, tool calling, memory, and evaluation, or create your own. Add loops, branches, and conditional logic to precisely control how context moves through your pipelines and agent workflows.

Extensible ecosystem
Build and share custom components through a consistent interface that makes it easy for the community and third parties to extend Haystack and contribute to an open ecosystem.

Tip

Would you like to deploy and serve Haystack pipelines as REST APIs or MCP servers? Hayhooks provides a simple way for you to wrap pipelines and agents with custom logic and expose them through HTTP endpoints or MCP. It also supports OpenAI-compatible chat completion endpoints and works with chat UIs like open-webui.

Haystack Enterprise: Support & Platform

Get expert support from the Haystack team, build faster with enterprise-grade templates, and scale securely with deployment guides for cloud and on-prem environments with Haystack Enterprise Starter. Read more about it in the announcement post.

๐Ÿ‘‰ Get Haystack Enterprise Starter

Need a managed production setup for Haystack? The Haystack Enterprise Platform helps you build, test, deploy and operate Haystack pipelines with built-in observability, collaboration, governance, and access controls. Itโ€™s available as a managed cloud service or as a self-hosted solution.

๐Ÿ‘‰ Learn more about Haystack Enterprise Platform or try it free

Telemetry

Haystack collects anonymous usage statistics of pipeline components. We receive an event every time these components are initialized. This way, we know which components are most relevant to our community.

Read more about telemetry in Haystack or how you can opt out in Haystack docs.

๐Ÿ–– Community

If you have a feature request or a bug report, feel free to open an issue in GitHub. We regularly check these, so you can expect a quick response. If you'd like to discuss a topic or get more general advice on how to make Haystack work for your project, you can start a thread in Github Discussions or our Discord channel. We also check ๐• (Twitter) and Stack Overflow.

Contributing to Haystack

We are very open to the community's contributions - be it a quick fix of a typo, or a completely new feature! You don't need to be a Haystack expert to provide meaningful improvements. To learn how to get started, check out our Contributor Guidelines first.

There are several ways you can contribute to Haystack:

Organizations using Haystack

Haystack is used by thousands of teams building production AI systems across industries, including:

Are you also using Haystack? Open a PR or tell us your story

Release History

VersionChangesUrgencyDate
v2.30.0## โญ๏ธ Highlights ### ๐Ÿ Syntax-aware Python code splitting with `PythonCodeSplitter` The new `PythonCodeSplitter` is a syntax-aware splitter for Python source files, built for code-RAG and code-search pipelines where naive line-based splitting tends to cut through functions and lose structural context. It parses sources with the `ast` module and greedily merges units, such as module docstring, import blocks, top-level functions, class headers, methods, and nested classes, into chunks of roHigh6/3/2026
v2.29.0## โญ๏ธ Highlights ### ๐Ÿ” Combine Retrievers with `MultiRetriever` and `TextEmbeddingRetriever` Two new retriever components make it easier to build hybrid search pipelines. `MultiRetriever` runs multiple text retrievers in parallel and merges their results into a single deduplicated list, ranked by reciprocal rank fusion by default. You can selectively enable or disable individual retrievers at runtime using the `active_retrievers` parameter. This is useful when you want to skip the embeddiHigh5/12/2026
v2.28.0# Release Notes ## v2.28.0 ### Upgrade Notes - As part of the migration from `requests` to `httpx`, `request_with_retry` and `async_request_with_retry` (in `haystack.utils.requests_utils`) no longer raise `requests.exceptions.RequestException` on failure; they now raise `httpx.HTTPError` instead. This also affects `HuggingFaceTEIRanker`, which relies on these utilities. Users catching `requests.exceptions.RequestException` should update their code to catch `httpx.HTTPError`. - The `LLM` compHigh4/20/2026
v2.28.0-rc2# Release Notes ## ๐Ÿš€ New Features - Tools and components can now declare a `State` (or `State | None`) parameter in their signature to receive the live agent `State` object at invocation time โ€” no extra wiring needed. For function-based tools created with `@tool` or `create_tool_from_function`, add a `state` parameter annotated as `State`: ``` python from haystack.components.agents import State from haystack.tools import tool @tool def my_tool(query: str, state: StHigh4/20/2026
v2.28.0-rc1# Release Notes ## v2.28.0-rc1 ## ๐Ÿš€ New Features - Tools and components can now declare a `State` (or `State | None`) parameter in their signature to receive the live agent `State` object at invocation time โ€” no extra wiring needed. For function-based tools created with `@tool` or `create_tool_from_function`, add a `state` parameter annotated as `State`: ``` python from haystack.components.agents import State from haystack.tools import tool @tool def my_tool(queHigh4/20/2026
v2.27.0## โญ๏ธ Highlights ### ๐Ÿ”Œ Automatic List Joining in Pipeline When a component expects a list as input, pipelines now automatically join multiple inputs into that list (no extra components needed), even if they come in different but compatible types. This enables patterns like combining a plain query string with a list of `ChatMessage` objects into a single `list[ChatMessage]` input. Supported conversations: Source Types | Target Type | Behavior -- | -- | -- T + T | list[T] | CombinesMedium4/1/2026
v2.27.0-rc1Release v2.27.0-rc1Medium3/31/2026
v2.26.1## Security Notes - Fixed an issue in `ChatPromptBuilder` where specially crafted template variables could be interpreted as structured content (e.g., images, tool calls) instead of plain text. Template variables are now automatically sanitized during rendering, ensuring they are always treated as plain text. ## ๐Ÿ› Bug Fixes - Fix malformed log format string in `DocumentCleaner`. The warning for documents with `None` content used `%{document_id}` instead of `{document_id}`, preventing pLow3/20/2026
v2.26.1-rc1Release v2.26.1-rc1Low3/19/2026
v2.26.0## โญ Highlights ### ๐Ÿง  More Flexible Agents with Dynamic System Prompts `Agent` now supports Jinja2 templating in `system_prompt`, enabling runtime parameter injection and conditional logic directly in system messages. This makes it easier to adapt agent behavior dynamically (e.g. language, tone, time-aware responses) and reuse agents across contexts without redefining prompts ``` python from haystack.components.agents import Agent from haystack.components.generators.chat import OpenAICLow3/18/2026
v2.26.0-rc1Release v2.26.0-rc1Low3/16/2026
v2.25.2## ๐Ÿ› Bug Fixes - Reverts the change that made Agent messages optional as it caused issues with pipeline execution. As a consequence, the LLM component now defaults to an empty messages list unless provided at runtime. Low3/5/2026
v2.25.2-rc1Release v2.25.2-rc1Low3/5/2026
v2.25.1## โšก๏ธ Enhancement Notes - Auto variadic sockets now also support `Optional[list[...]]` input types, in addition to plain `list[...]`. ## ๐Ÿ› Bug Fixes - Fixed smart connection logic to support connecting multiple outputs to a socket whose type is `Optional[list[...]]` (e.g. `list[ChatMessage] | None`). Previously, connecting two `list[ChatMessage]` outputs to `Agent.messages` would fail after its type was updated from `list[ChatMessage]` to `list[ChatMessage] | None`. Low2/27/2026
v2.25.1-rc1Release v2.25.1-rc1Low2/27/2026
v2.25.0## โญ๏ธ Highlights ### ๐Ÿ› ๏ธ Dynamic Tool Discovery with `SearchableToolset` For applications with large tool catalogs, weโ€™ve added the [SearchableToolset](https://docs.haystack.deepset.ai/docs/searchabletoolset). Instead of exposing all tools upfront, agents start with a single `search_tools` function and dynamically discover relevant tools using BM25-based keyword search. This is particularly useful when connecting MCP servers via [MCPToolset](https://docs.haystack.deepset.ai/docs/mcptoolset)Low2/26/2026
v2.25.0-rc1Release v2.25.0-rc1Low2/25/2026
v2.24.1## ๐Ÿ› Bug Fixes - Fixed a bug in flexible Pipeline connections that prevented automatic value conversion when the receiving component expects a Union type. For example, connecting a component returning `ChatMessage` to a receiver expecting `list[str] | list[ChatMessage]` should have worked but did not. The conversion strategy now correctly evaluates each branch of a Union receiver and picks the best match. Low2/12/2026
v2.24.1-rc1Release v2.24.1-rc1Low2/12/2026
v2.24.0## โญ๏ธ Highlights ### ๐Ÿ”Œ Pipelines got simpler With the updated logic, Pipelines can now: * connect multiple `list[T]` component outputs directly to a single `list[T]` input of the next component, simplifying pipeline definitions when multiple components produce compatible outputs. E.g., you can directly connect multiple converters to a writer component without a `DocumentJoiner` component in ingestion pipelines. * support the automatic conversion between `ChatMessage` and `str` types, enaLow2/12/2026
v2.24.0-rc2Release v2.24.0-rc2Low2/11/2026
v2.24.0-rc1Release v2.24.0-rc1Low2/10/2026
v2.23.0## โญ๏ธ Highlights ### ๐Ÿ”„ Human-in-the-Loop for Agents Agents can now pause for human confirmation before executing tools. You can define confirmation behavior per tool: **always ask**, **ask only on first use**, or **never ask** and fully customize the confirmation UI. This makes it much easier to build safer, more transparent agent workflows, especially when tools trigger side effects or access sensitive data. ``` python agent = Agent( chat_generator=OpenAIChatGenerator(model="gptLow1/27/2026
v2.23.0-rc1Release v2.23.0-rc1Low1/26/2026
v2.22.0## โญ๏ธ Highlights ### โœ‚๏ธ Smarter Document Chunking with Embedding-Based Splitting Introducing the new [EmbeddingBasedDocumentSplitter](https://docs.haystack.deepset.ai/docs/embeddingbaseddocumentsplitter), a component that takes an embedder and splits documents based on semantic similarity rather than fixed sizes or rules. ```python from haystack.components.embedders import SentenceTransformersDocumentEmbedder from haystack.components.preprocessors import EmbeddingBasedDocumentSplitter Low1/8/2026
v2.22.0-rc1Release v2.22.0-rc1Low1/7/2026
v2.21.0## โญ๏ธ Highlights ### ๐Ÿ” Smarter, Broader Retrieval with Multi-Query RAG This release introduces three new components that significantly boost retrieval recall in RAG systems by expanding the user query and retrieving documents across multiple reformulations: - `QueryExpander` generates semantically similar variations of a user query to broaden search coverage. - `MultiQueryTextRetriever` runs multiple queries in parallel using a text-based retriever (e.g., BM25) and merges results by sLow12/8/2025
v2.21.0-rc1Release v2.21.0-rc1Low12/3/2025
v2.20.0## โญ๏ธ Highlights ### Support for OpenAI's Responses API Haystack now integrates the **OpenAI's Responses API** through the new `OpenAIResponsesChatGenerator` and `AzureOpenAIResponsesChatGenerator` components. This unlocks several advanced capabilities like: * Retrieving concise summaries of the modelโ€™s reasoning process. * Using native OpenAI or MCP tool formats alongside Haystack `Tool` objects and `Toolset` instances. Example with reasoning and a web search tool: ```pythonLow11/13/2025
v2.20.0-rc2Release v2.20.0-rc2Low11/13/2025
v2.20.0-rc1Release v2.20.0-rc1Low11/11/2025
v2.19.0## โญ๏ธ Highlights ### ๐Ÿ›ก๏ธ Try Multiple LLMs with `FallbackChatGenerator` Introduced [`FallbackChatGenerator`](https://docs.haystack.deepset.ai/docs/fallbackchatgenerator), a resilient chat generator that runs multiple LLMs sequentially and automatically falls back when one fails. It tries each generator in order until one succeeds, handling errors like timeouts, rate limits, or server issues. Ideal for building robust, production-grade chat systems that stay responsive across providers. ``Low10/20/2025
v2.19.0-rc1Release v2.19.0-rc1Low10/20/2025
v2.18.1# Release Notes ## v2.18.1 ## โšก๏ธ Enhancement Notes - Added <span class="title-ref">tools</span> to agent run parameters to enhance the agent's flexibility. Users can now choose a subset of tools for the agent at runtime by providing a list of tool names, or supply an entirely new set by passing Tool objects or a Toolset. ## ๐Ÿ› Bug Fixes - Fix Agent `run_async` method to correctly handle async streaming callbacks. This previously triggered errors due to a bug. - Prevent duplLow9/29/2025
v2.18.0## โญ๏ธ Highlights ### ๐Ÿ” **Pipeline Error Recovery with Snapshots** Pipelines now capture a snapshot of the last successful step when a run fails, including intermediate outputs. This lets you diagnose issues (e.g., failed tool calls), fix them, and resume from the checkpoint instead of restarting the entire run. Currently supported for synchronous `Pipeline` and `Agent` (not yet in `AsyncPipeline`) The snapshot is part of the exception raised with the `PipelineRuntimeError` when the pipLow9/22/2025
v2.18.0-rc1Release v2.18.0-rc1Low9/17/2025
v2.17.1# Release Notes ## v2.17.1 ### Bug Fixes - Fixed the `from_dict` method of `MetadataRouter` so the `output_type` parameter introduced in Haystack 2.17 is now optional when loading from YAML. This ensures compatibility with older Haystack pipelines. - In `OpenAIChatGenerator`, improved the logic to exclude unsupported custom tool calls. The previous implementation caused compatibility issues with the Mistral Haystack core integration, which extends `OpenAIChatGenerator`. Low8/20/2025
v2.17.0## โญ๏ธ Highlights ### ๐Ÿ–ผ๏ธ Image support for several model providers Following the introduction of image support in Haystack 2.16.0, we've expanded this to more model providers in Haystack and Haystack Core integrations. **Now supported**: Amazon Bedrock, Anthropic, Azure, Google, Hugging Face API, Meta Llama API, Mistral, Nvidia, Ollama, OpenAI, OpenRouter, STACKIT. ### ๐Ÿงฉ Extended components We've improved several components to make them more flexible: - `MetadataRouter`, whichLow8/19/2025
v2.17.0-rc2Release v2.17.0-rc2Low8/19/2025
v2.17.0-rc1Release v2.17.0-rc1Low8/19/2025
v2.16.1# Release Notes ## v2.16.1 ### Bug Fixes - Improved validation in the `ChatMessage.from_user` class method. The method now raises an error if neither `text` nor `content_parts` are provided. It does not raise an error if `text` is an empty string. Low7/29/2025
v2.16.1-rc1Release v2.16.1-rc1Low7/29/2025
v2.16.0## โญ๏ธ Highlights ### ๐Ÿง  Agent Breakpoints This release introduces **Agent Breakpoints**, a powerful new feature that enhances debugging and observability when working with Haystack Agents. You can pause execution mid-run by inserting breakpoints in the Agent or its tools to inspect internal state and resume execution seamlessly. This brings fine-grained control to agent development and significantly improves traceability during complex interactions. ```python from haystack.dataclasses.Low7/29/2025
v2.16.0-rc1Release v2.16.0-rc1Low7/24/2025
v2.15.2### Enhancement Notes - Weโ€™ve relaxed the requirements for the `ToolCallDelta` dataclass (introduced in Haystack 2.15). Previously, creating a `ToolCallDelta` instance required either the parameters argument or the name to be set. This constraint has now been removed to align more closely with OpenAI's SDK behavior. The change was necessary as the stricter requirement was causing errors in certain hosted versions of open-source models that adhere to the OpenAI SDK specification. ### BugLow7/4/2025
v2.15.2-rc1Release v2.15.2-rc1Low7/4/2025
v2.15.1### Bug Fixes - Fix `_convert_streaming_chunks_to_chat_message` which is used to convert Haystack `StreamingChunks` into a Haystack `ChatMessage`. This fixes the scenario where one StreamingChunk contains two `ToolCallDetlas` in StreamingChunk.tool_calls. With this fix this correctly saves both `ToolCallDeltas` whereas before they were overwriting each other. This only occurs with some LLM providers like Mistral (and not OpenAI) due to how the provider returns tool calls.Low6/30/2025
v2.15.1-rc1 Low6/30/2025
v2.15.0## โญ๏ธ Highlights ### Parallel Tool Calling for Faster Agents - `ToolInvoker` now processes all tool calls passed to `run` or `run_async` in parallel using an internal `ThreadPoolExecutor`. This improves performance by reducing the time spent on sequential tool invocations. - This parallel execution capability enables `ToolInvoker` to batch and process multiple tool calls concurrently, allowing Agents to run complex pipelines efficiently with decreased latency. - You no longer need to pasLow6/26/2025
v2.15.0-rc1Release v2.15.0-rc1Low6/25/2025
v2.14.3### Bug Fixes - In ConditionalRouter fixed the `to_dict` and `from_dict` methods to properly handle the case when `output_type` is a List of types or a List of strings. This occurs when a user specifies a route in ConditionalRouter to have multiple outputs. - Fix the serialization of ComponentTool and Tool when specifying `outputs_to_string`. Previously an error occurred on deserialization right after serializing if `outputs_to_string` is not None. Low6/19/2025
v2.14.3-rc1Release v2.14.3-rc1Low6/18/2025

Dependencies & License Audit

Loading dependencies...

Similar Packages

langroidHarness LLMs with Multi-Agent Programming0.65.0
langgraphjsFramework to build resilient language agents as graphs.@langchain/svelte@1.0.16
txtai๐Ÿ’ก All-in-one AI framework for semantic search, LLM orchestration and language model workflowsv9.10.0
langgraphBuild resilient language agents as graphs.1.2.4
arkaosThe Operating System for AI Agent Teamsv4.0.1

More from deepset-ai

haystack-cookbook๐Ÿ‘ฉ๐Ÿปโ€๐Ÿณ A collection of example notebooks using Haystack

More in Frameworks

langchainThe agent engineering platform
deer-flowAn open-source long-horizon SuperAgent harness that researches, codes, and creates. With the help of sandboxes, memories, tools, skill, subagents and message gateway, it handles different levels of ta
tqdmFast, Extensible Progress Meter
simBuild, deploy, and orchestrate AI agents. Sim is the central intelligence layer for your AI workforce.