NeMo Guardrails is a toolkit for adding programmable guardrails to LLM-based conversational applications. A guardrail, or rail, is a rule that controls model behaviour: keeping a chatbot off certain topics, responding in a set way to specific requests, following a predefined dialog path, using a particular style, or extracting structured data. The rails sit as a layer between your application code and the LLM.
You load a guardrails configuration into an LLMRails instance and call generate or generate_async with chat-style messages in place of calling the model directly; the toolkit is async-first, and every public method has sync and async forms. A configuration folder holds a config.yml naming the models and active rails, optional Python actions, and .co files written in Colang, a Python-like modelling language for dialogue flows. Colang 1.0 is the default and Colang 2.0 is also supported. A guardrails server is available as an alternative to the Python API, and a LangChain integration wraps chains.
Use cases include fact-checking and output moderation for RAG question answering, keeping domain-specific assistants on topic, and protecting custom LLM endpoints against jailbreaks and prompt injections. Supported models include OpenAI GPT models, LLaMa-2, Falcon, Vicuna, and Mosaic.
Features
- Input rails: reject or alter user input, for example masking sensitive data or rephrasing
- Dialog rails: operate on canonical-form messages to decide whether to run an action, prompt the LLM, or return a predefined response
- Retrieval rails: filter or modify retrieved chunks before they reach the prompt in RAG scenarios
- Execution rails: check the input and output of custom actions (tools) the LLM calls
- Output rails: block or edit generated responses, such as removing sensitive data
- Colang: a dedicated language for defining user intents, bot responses, and conversational flows, in versions 1.0 and 2.0
- Built-in rails library: ready-made rails such as jailbreak checks, self-check facts and hallucination, sensitive-data detection, and third-party moderation
- Vulnerability scanning: evaluation tooling for measuring protection against jailbreaks and prompt injection on a sample bot
- Deployment options: Python API or a guardrails server, with an OpenAI Chat Completions-style message format
- LangChain integration: a guardrails layer around chains via setdefaultframework("langchain")
