Langchain-Chatchat is a question-answering application built over local knowledge bases, designed to run offline and to work well with Chinese-language material and open-source models. Its pipeline loads files, reads and splits the text, embeds the chunks, embeds the question, matches the top k most similar chunks, and passes those passages to a language model as context alongside the question.
From version 0.3.0 the project no longer loads model weights itself. You start a separate inference framework — Xinference, Ollama, LocalAI, or FastChat — and connect it through configuration, and the One API gateway covers hosted services such as OpenAI, Azure OpenAI, Anthropic Claude, Zhipu, and Baichuan. All model access is compatible with the OpenAI SDK. You reach the application through a FastAPI-based HTTP API or a Streamlit web UI, and configuration lives in local YAML files that the server picks up without a restart.
Version 0.3.x builds its core features around an agent loop, with tool calling optimised for ChatGLM3 and Qwen. Alongside plain chat and knowledge-base chat it adds database chat, multimodal image chat, arXiv and Wolfram queries, and text-to-image generation, and file chat is unified into a File RAG feature that supports BM25 and KNN retrieval rather than vector search alone. You can enable the agent and let the model select among several tools, enable it with a single tool so the model only parses arguments, or disable it and fill in the arguments yourself when the model has no tool-calling ability.
The application runs on Python 3.8 to 3.11 and is tested on Windows, macOS, and Linux. Because model loading is delegated, it works on CPU, GPU, NPU, or MPS hardware. Fine-tuning and training are outside its scope, although a fine-tuned model can be connected to it.
Features
- Local knowledge-base QA: documents are chunked, embedded, and retrieved by similarity, then answered with the retrieved passages as context
- Offline deployment: open-source LLM, embedding, and vector-database components allow a fully private, disconnected installation
- Pluggable inference frameworks: Xinference, LocalAI, Ollama, and FastChat are connected through configuration instead of direct weight loading
- Hosted-API access: the One API gateway adds OpenAI, Azure OpenAI, Anthropic Claude, Zhipu, and Baichuan endpoints
- File RAG: BM25 and KNN retrieval modes in addition to vector search
- Database chat: natural-language querying of a configured database
- Multimodal chat: image conversation with vision models such as qwen-vl-chat
- Reference tools: arXiv literature lookup, Wolfram queries, search-engine chat, and text-to-image generation
- Two front ends: a FastAPI HTTP API and a Streamlit web UI with multi-session support and custom system prompts
- Live YAML configuration: model, knowledge-base, and path settings are edited in local files and reloaded without restarting
