Langroid

Langroid is a Python framework (Python 3.11+) for building LLM applications as collaborating agents. You set up agents, give each one an optional LLM, vector store, and set of tools or functions, assign them tasks, and let them solve a problem by exchanging messages. The multi-agent design draws on the actor model, and the framework does not depend on LangChain or any other LLM framework.

Two abstractions carry the design. An Agent encapsulates conversation state plus its optional vector store and tools, and acts as a message transformer with three responder methods, one each for the LLM, the agent itself, and the user. A Task wraps an agent with instructions, iterates over its responders, and orchestrates multi-agent work through hierarchical, recursive delegation: Task.run() shares the responder type signature, so from a task's point of view its sub-tasks are additional responders consulted round-robin after the agent's own.

Tools and function calling share one Pydantic-based interface: you define a ToolMessage class rather than a JSON schema, and it works with OpenAI function calling or with any other LLM through an equivalent mechanism. When a model emits malformed JSON, the Pydantic error is returned to the model so it can correct itself. An MCP adapter converts a server's tools into ToolMessage instances, and structured output enforces strict JSON schemas on compatible models.

Beyond OpenAI, the framework talks to local and remote models through OpenAI-compatible servers and proxies such as Ollama, oobabooga, LiteLLM, OpenRouter, Portkey, LangDB, Cerebras, glhf.chat, Gemini, and Azure OpenAI. Reasoning content from models such as DeepSeek-R1 and o1 can be captured and streamed alongside the final answer, and PDF and image inputs are accepted as multimodal messages.

Retrieval-augmented generation is handled by DocChatAgent, which grounds answers in documents from a vector store and cites sources; it fuses chunks retrieved by different methods with reciprocal rank fusion, enriches chunks with section headers, and can rerank with a cross-encoder. Supported vector stores are Qdrant, Chroma, LanceDB, Pinecone, PGVector, Weaviate, and Milvus, with ArangoDB for knowledge graphs. Document parsers include Docling, Marker, pymupdf4llm, Markitdown, and LLM-based PDF parsing, and web content arrives through Crawl4AI, Firecrawl, Exa, and Tavily. SQLChatAgent covers chat with databases.

Features

• Agents and tasks: agents hold LLM, vector store, and tools; tasks wrap them and delegate recursively to sub-tasks
• TaskTool: an agent spawns sub-agents at run time with their own tools and configuration
• Declarative termination: donesequences end a task on event patterns, and maxtime budgets end it on elapsed time
• Pydantic tools: one definition serves OpenAI function calling and any other LLM; validation errors are fed back to the model
• MCP support: server tools become ToolMessage instances, with namespacing across multiple servers
• Model coverage: OpenAI, Gemini, Azure OpenAI, and any OpenAI-compatible endpoint, local or hosted
• RAG: DocChatAgent with reciprocal rank fusion, chunk enrichment, reranking, and source citation across seven vector stores
• Document and web ingestion: Docling, Marker, pymupdf4llm, Markitdown, Crawl4AI, Firecrawl, Exa, and Tavily
• Observability: detailed multi-agent logs, message lineage tracking, an HTML task logger, and portable JSON chat-history snapshots
• Security controls: tool-origin taint tracking, code-injection protection, and per-provider env_prefix for vector-store configs
• Caching and batching: Redis-backed LLM response caching and runbatchtask for parallel runs
• Claude Code plugin: langroid:patterns and langroid:add-pattern skills generate and record multi-agent code patterns