llm-chain is a collection of Rust crates for building LLM applications such as chatbots and agents. It works with both cloud-hosted and locally hosted models.
Its core abstractions are prompt templates and chains. A chain runs a sequence of prompts so that a task too large for one model call is split into steps. Agents built with it can be given tools, such as running Bash commands, executing Python scripts, or performing web searches. Vector store integrations give a model long-term memory and domain knowledge.
Model support is split across crates: llm-chain-openai covers OpenAI's ChatGPT models, and other crates cover LLaMA, Alpaca, and llm.rs, which runs models in Rust with no dependency on C++ code. It requires Rust 1.65.0 or later, and a template project is available for starting a new application.
Features
- Prompt templates: reusable, parameterized prompts for structured model interactions
- Chains: multi-step sequences of prompts for tasks that do not fit a single call
- Tools: agent access to Bash commands, Python execution, and web search
- Vector stores: integrations that give a model long-term memory and domain knowledge
- OpenAI support: ChatGPT models through the llm-chain-openai crate
- Local models: LLaMA and Alpaca models, plus llm.rs for running models in pure Rust
- Extensible executors: additional LLM backends can be added as separate crates
