This site is not available on Mobile. Please return on a desktop browser.
Visit our main site at guardrailsai.com
This site is not available on Mobile. Please return on a desktop browser.
Visit our main site at guardrailsai.com
| Developed by | Guardrails AI |
|---|---|
| Date of development | Feb 15, 2024 |
| Validator type | RAG |
| Blog | https://www.guardrailsai.com/blog/reduce-ai-hallucinations-provenance-guardrails |
| License | Apache 2 |
| Input/Output | Output |
This validator uses embedding similarity to check if an LLM generated text can be supported by the sources. In order for this validator to work, a list of sources must be provided to which the LLM generated text can be attributed. In order to use the validator, you must provide either of the following in the metadata:
query_function: The query_function function should take a string as input (the LLM-generated text) and return a list of relevant chunks. The list should be sorted in ascending order by the distance between the chunk and the LLM-generated text.sources and embed_function: The embed_function should take a string or a list of strings as input and return a np array of floats. The vector should be normalized to unit length.Below is a step-wise breakdown of how the validator works:
k nearest source chunks. If the cosine distance is less than a certain threshold, then the LLM-generated text is considered attributable to the source text.The primary intended uses are for RAG applications -- to check if a text is hallucinated by establishing a source (i.e. provenance) for any LLM generated text. Out of scope use cases are general question answering without RAG or text grounding. You can use a combination of RAG and this validator on the output to implement Retrieval-Augmented Validated Generation (RAVG).
Dependencies:
numpynltkTo use in an example:
sentence-transformerschromadbFoundation model access keys:
$ guardrails hub install hub://guardrails/provenance_embeddings
In this example, we apply the validator to a string output generated by an LLM.
# Import Guard and Validator
from guardrails.hub import ProvenanceEmbeddings
from guardrails import Guard
import numpy as np
try:
from sentence_transformers import SentenceTransformer
except ImportError:
raise ImportError(
"This example requires the `sentence-transformers` package. "
"Install it with `pip install sentence-transformers`, and try again."
)
# Setup text sources
SOURCES = [
"The sun is a star.",
"The sun rises in the east and sets in the west.",
"Sun is the largest object in the solar system, and all planets revolve around it.",
]
# Load model for embedding function
MODEL = SentenceTransformer("paraphrase-MiniLM-L6-v2")
# Create embed function
def embed_function(sources: list[str]) -> np.array:
return MODEL.encode(sources)
# Use the Guard with the validator
guard = Guard().use(
ProvenanceEmbeddings,
threshold=0.2, # Lower the threshold to make the validator stricter
validation_method="sentence",
on_fail="exception",
)
# Test passing response
guard.validate(
"""
The sun is a star that rises in the east and sets in the west.
""",
metadata={"sources": SOURCES, "embed_function": embed_function},
)
try:
# Test failing response
guard.validate(
"""
Pluto is the farthest planet from the sun.
""", # This sentence is not "false", but is still NOT supported by the sources
metadata={"sources": SOURCES, "embed_function": embed_function},
)
except Exception as e:
print(e)
Output:
Validation failed for field with errors: None of the following sentences in your response are supported by provided context:
- Pluto is the farthest planet from the sun.
__init__(self, threshold=0.8, validation_method="sentence", on_fail="noop")
Initializes a new instance of the Validator class.
Parameters:
threshold (float): The minimum cosine distance between the generated text and the source text. Defaults to 0.8.validation_method (str): Whether to validate at the sentence level or over the full text. Must be one of sentence or full. Defaults to sentenceon_fail (str, Callable): The policy to enact when a validator fails. If str, must be one of reask, fix, filter, refrain, noop, exception or fix_reask. Otherwise, must be a function that is called when the validator fails.__call__(self, value, metadata={}) -> ValidationResult
Validates the given value using the rules defined in this validator, relying on the metadata provided to customize the validation process. This method is automatically invoked by guard.parse(...), ensuring the validation logic is applied to the input data.
Note:
guard.parse(...) where this method will be called internally for each associated Validator.guard.parse(...), ensure to pass the appropriate metadata dictionary that includes keys and values required by this validator. If guard is associated with multiple validators, combine all necessary metadata into a single dictionary.Parameters:
value (Any): The input value to validate.
metadata (dict): A dictionary containing metadata required for validation. Keys and values must match the expectations of this validator.
| Key | Type | Description | Default |
|---|---|---|---|
query_function | Optional[Callable] | A callable that takes a string and returns a list of (chunk, score) tuples. In order to use this validator, you must provide either a query_function or sources with an embed_function in the metadata. The query_function should take a string as input and return a list of (chunk, score) tuples. The chunk is a string and the score is a float representing the cosine distance between the chunk and the input string. The list should be sorted in ascending order by score. | None |
sources | Optional[List[str]] | The source text. In order to use this validator, you must provide either a query_function or sources with an embed_function in the metadata. | None |
embed_function | Optional[Callable] | A callable that creates embeddings for the sources. Must accept a list of strings and return an np.array of floats. | sentence-transformer's paraphrase-MiniLM-L6-v2 |
The validator playground is available to authenticated users. Please log in to use it.