nvidia-nemo/guardrailsProgrammable guardrails for LLM chat apps: enforce input/output rails, block jailbreaks and prompt injections, detect hallucination, and mask sensitive data.
LATEST RELEASE / DEVELOPMENT VERSION: The develop branch tracks the latest top of tree development. The latest released version is 0.24.1.
✨✨✨
📌 The official NeMo Guardrails library documentation is available at docs.nvidia.com/nemo/guardrails.
✨✨✨
NVIDIA NeMo Guardrails library is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational applications. Guardrails (or "rails" for short) are specific ways of controlling the output of a large language model, such as not talking about politics, responding in a particular way to specific user requests, following a predefined dialog path, using a particular language style, extracting structured data, and more.
This paper introduces the NeMo Guardrails library and contains a technical overview of the system and the current evaluation.
Python 3.11, 3.12 or 3.13.
Starting with version 0.25.0, Python 3.10 is no longer supported. Upgrade your Python environment before upgrading the library.
To install using pip:
> pip install nemoguardrails
For more detailed instructions, see the Installation Guide.
The NeMo Guardrails library enables developers building LLM-based applications to add programmable guardrails between the application code and the LLM.
Key benefits of adding programmable guardrails include:
Building Trustworthy, Safe, and Secure LLM-based Applications: you can define rails to guide and safeguard conversations; you can choose to define the behavior of your LLM-based application on specific topics and prevent it from engaging in discussions on unwanted topics.
Connecting models, chains, and other services securely: you can connect an LLM to other services (a.k.a. tools) seamlessly and securely.
Controllable dialog: you can steer the LLM to follow pre-defined conversational paths, allowing you to design the interaction following conversation design best practices and enforce standard operating procedures (e.g., authentication, support).
The NeMo Guardrails library provides several mechanisms for protecting an LLM-powered chat application against common LLM vulnerabilities, such as jailbreaks and prompt injections. Below is a sample overview of the protection offered by different guardrails configuration for the example ABC Bot included in this repository. For more details, please refer to the LLM Vulnerability Scanning page.
You can use programmable guardrails in different types of use cases:
NEMOGUARDRAILS_LLM_FRAMEWORK=langchain environment variable or call set_default_framework("langchain").To add programmable guardrails to your application you can use the Python API or a guardrails server (see the Server Guide for more details). Using the Python API is similar to using the LLM directly. Calling the guardrails layer instead of the LLM requires only minimal changes to the code base, and it involves two simple steps:
LLMRails instance.generate/generate_async methods.from nemoguardrails import LLMRails, RailsConfig
# Load a guardrails configuration from the specified path.
config = RailsConfig.from_path("PATH/TO/CONFIG")
rails = LLMRails(config)
completion = rails.generate(
messages=[{"role": "user", "content": "Hello world!"}]
)
Sample output:
{"role": "assistant", "content": "Hi! How can I help you?"}
The input and output format for the generate method is similar to the Chat Completions API from OpenAI.
The NeMo Guardrails library is an async-first toolkit as the core mechanics are implemented using the Python async model. The public methods have both a sync and an async version. For example: LLMRails.generate and LLMRails.generate_async.
You can use NeMo Guardrails with multiple LLMs like OpenAI GPT-3.5, GPT-4, LLaMa-2, Falcon, Vicuna, or Mosaic. For more details, check out the Supported LLM Models section in the Configuration Guide.
The NeMo Guardrails library supports five main types of guardrails:
