Simple techniques to alleviate LLM hallucinations using LangGraph<\/h4>\n![\"\"](\"https:\/\/cdn-images-1.medium.com\/max\/1024\/0*fNRfvA4TOQuHOzfh\")
Photo by Greg Rakozy<\/a> on\u00a0Unsplash<\/a><\/figcaption><\/figure>\nIf you\u2019ve worked with LLMs, you know they can sometimes hallucinate. This means they generate text that\u2019s either nonsensical or contradicts the input data. It\u2019s a common issue that can hurts the reliability of LLM-powered applications.<\/p>\n
In this post, we\u2019ll explore a few simple techniques to reduce the likelihood of hallucinations. By following these tips, you can (hopefully) improve the accuracy of your AI applications.<\/p>\n
There are multiple types of hallucinations:<\/p>\n
\n- \nIntrinsic hallucinations<\/a>: the LLM\u2019s response contradicts the user-provided context. This is when the response is verifiably wrong withing the current\u00a0context.<\/li>\n
- \nExtrinsic hallucinations<\/a>: the LLM\u2019s response cannot be verified using the user-provided context. This is when the response may or may not be wrong but we have no way of confirming that using the current\u00a0context.<\/li>\n
- Incoherent hallucinations: the LLM\u2019s response does not answer the question or does not make sense. This is when the LLM is unable to follow the instructions.<\/li>\n<\/ul>\n
In this post, we will target all the types mentioned above.<\/p>\n
We will list out a set of tips and tricks that work in different ways in reducing hallucinations.<\/p>\n
Tip 1: Use Grounding<\/h4>\n
Grounding is using in-domain relevant additional context in the input of the LLM when asking it to do a task. This gives the LLM the information it needs to correctly answer the question and reduces the likelihood of a hallucination. This is one the reason we use Retrieval augmented generation (RAG).<\/p>\n
For example asking the LLM a math question OR asking it the same question while providing it with relevant sections of a math book will yield different results, with the second option being more likely to be\u00a0right.<\/p>\n
Here is an example of such implementation in one of my previous tutorials where I provide document-extracted context when asking a question:<\/p>\n
Build a Document AI pipeline for ANY type of PDF With Gemini<\/a><\/p>\nTip 2: Use structured outputs<\/h4>\n
Using structured outputs means forcing the LLM to output valid JSON or YAML text. This will allow you to reduce the useless ramblings and get \u201cstraight-to-the-point\u201d answers about what you need from the LLM. It also will help with the next tips as it makes the LLM responses easier to\u00a0verify.<\/p>\n
Here is how you can do this with Gemini\u2019s\u00a0API:<\/p>\n
import json
import google.generativeai as genai
from pydantic import BaseModel, Field
from document_ai_agents.schema_utils import prepare_schema_for_gemini
class Answer(BaseModel):
answer: str = Field(..., description=\"Your Answer.\")
model = genai.GenerativeModel(\"gemini-1.5-flash-002\")
answer_schema = prepare_schema_for_gemini(Answer)
question = \"List all the reasons why LLM hallucinate\"
context = (
\"LLM hallucination refers to the phenomenon where large language models generate plausible-sounding but\"
\" factually incorrect or nonsensical information. This can occur due to various factors, including biases\"
\" in the training data, the inherent limitations of the model's understanding of the real world, and the \"
\"model's tendency to prioritize fluency and coherence over accuracy.\"
)
messages = (
[context]
+ [
f\"Answer this question: {question}\",
]
+ [
f\"Use this schema for your answer: {answer_schema}\",
]
)
response = model.generate_content(
messages,
generation_config={
\"response_mime_type\": \"application\/json\",
\"response_schema\": answer_schema,
\"temperature\": 0.0,
},
)
response = Answer(**json.loads(response.text))
print(f\"{response.answer=}\")<\/pre>\nWhere \u201cprepare_schema_for_gemini\u201d is a utility function that prepares the schema to match Gemini\u2019s weird requirements. You can find its definition here:\u00a0code<\/a>.<\/p>\nThis code defines a Pydantic schema and sends this schema as part of the query in the field \u201cresponse_schema\u201d. This forces the LLM to follow this schema in its response and makes it easier to parse its\u00a0output.<\/p>\n
Tip 3: Use chain of thoughts and better prompting<\/h4>\n
Sometimes, giving the LLM the space to work out its response, before committing to a final answer, can help produce better quality responses. This technique is called Chain-of-thoughts and is widely used as it is effective and very easy to implement.<\/p>\n
We can also explicitly ask the LLM to answer with \u201cN\/A\u201d if it can\u2019t find enough context to produce a quality response. This will give it an easy way out instead of trying to respond to questions it has no answer\u00a0to.<\/p>\n
For example, lets look into this simple question and\u00a0context:<\/p>\n
Context<\/strong><\/p>\nThomas Jefferson (April 13 [O.S. April 2], 1743\u200a\u2014\u200aJuly 4, 1826) was an American statesman, planter, diplomat, lawyer, architect, philosopher, and Founding Father who served as the third president of the United States from 1801 to 1809.[6] He was the primary author of the Declaration of Independence. Following the American Revolutionary War and before becoming president in 1801, Jefferson was the nation\u2019s first U.S. secretary of state under George Washington and then the nation\u2019s second vice president under John Adams. Jefferson was a leading proponent of democracy, republicanism, and natural rights, and he produced formative documents and decisions at the state, national, and international levels. (Source: Wikipedia)<\/p><\/blockquote>\n
Question<\/strong><\/p>\nWhat year did davis jefferson die?<\/p><\/blockquote>\n
A naive approach\u00a0yields:<\/p>\n
Response<\/strong><\/p>\nanswer=\u20191826′<\/p><\/blockquote>\n
Which is obviously false as Jefferson Davis is not even mentioned in the context at all. It was Thomas Jefferson that died in\u00a01826.<\/p>\n
If we change the schema of the response to use chain-of-thoughts to:<\/p>\n
class AnswerChainOfThoughts(BaseModel):
rationale: str = Field(
...,
description=\"Justification of your answer.\",
)
answer: str = Field(
..., description=\"Your Answer. Answer with 'N\/A' if answer is not found\"
)<\/pre>\nWe are also adding more details about what we expect as output when the question is not answerable using the context \u201cAnswer with \u2018N\/A\u2019 if answer is not\u00a0found\u201d<\/p>\n
With this new approach, we get the following rationale<\/strong> (remember, chain-of-thought):<\/p>\nThe provided text discusses Thomas Jefferson, not Jefferson Davis. No information about the death of Jefferson Davis is included.<\/p><\/blockquote>\n
And the final\u00a0answer<\/strong>:<\/p>\nanswer=\u2019N\/A\u2019<\/p><\/blockquote>\n
Great\u00a0! But can we use a more general approach to hallucination detection?<\/p>\n
We can, with\u00a0Agents!<\/p>\n
Tip 4: Use an Agentic\u00a0approach<\/h4>\n
We will build a simple agent that implements a three-step process:<\/p>\n
\n- The first step is to include the context and ask the question to the LLM in order to get the first candidate response and the relevant context that it had used for its\u00a0answer.<\/li>\n
- The second step is to reformulate the question and the first candidate response as a declarative statement.<\/li>\n
- The third step is to ask the LLM to verify whether or not the relevant context entails<\/strong> the candidate response. It is called \u201cSelf-verification\u201d: https:\/\/arxiv.org\/pdf\/2212.09561<\/a>\n<\/li>\n<\/ul>\n
In order to implement this, we define three nodes in LangGraph. The first node will ask the question while including the context, the second node will reformulate it using the LLM and the third node will check the entailment of the statement in relation to the input\u00a0context.<\/p>\n
The first node can be defined as\u00a0follows:<\/p>\n
def answer_question(self, state: DocumentQAState):
logger.info(f\"Responding to question '{state.question}'\")
assert (
state.pages_as_base64_jpeg_images or state.pages_as_text
), \"Input text or images\"
messages = (
[
{\"mime_type\": \"image\/jpeg\", \"data\": base64_jpeg}
for base64_jpeg in state.pages_as_base64_jpeg_images
]
+ state.pages_as_text
+ [
f\"Answer this question: {state.question}\",
]
+ [
f\"Use this schema for your answer: {self.answer_cot_schema}\",
]
)
response = self.model.generate_content(
messages,
generation_config={
\"response_mime_type\": \"application\/json\",
\"response_schema\": self.answer_cot_schema,
\"temperature\": 0.0,
},
)
answer_cot = AnswerChainOfThoughts(**json.loads(response.text))
return {\"answer_cot\": answer_cot}<\/pre>\nAnd the second one\u00a0as:<\/p>\n
def reformulate_answer(self, state: DocumentQAState):
logger.info(\"Reformulating answer\")
if state.answer_cot.answer == \"N\/A\":
return
messages = [
{
\"role\": \"user\",
\"parts\": [
{
\"text\": \"Reformulate this question and its answer as a single assertion.\"
},
{\"text\": f\"Question: {state.question}\"},
{\"text\": f\"Answer: {state.answer_cot.answer}\"},
]
+ [
{
\"text\": f\"Use this schema for your answer: {self.declarative_answer_schema}\"
}
],
}
]
response = self.model.generate_content(
messages,
generation_config={
\"response_mime_type\": \"application\/json\",
\"response_schema\": self.declarative_answer_schema,
\"temperature\": 0.0,
},
)
answer_reformulation = AnswerReformulation(**json.loads(response.text))
return {\"answer_reformulation\": answer_reformulation}<\/pre>\nThe third one\u00a0as:<\/p>\n
def verify_answer(self, state: DocumentQAState):
logger.info(f\"Verifying answer '{state.answer_cot.answer}'\")
if state.answer_cot.answer == \"N\/A\":
return
messages = [
{
\"role\": \"user\",
\"parts\": [
{
\"text\": \"Analyse the following context and the assertion and decide whether the context \"
\"entails the assertion or not.\"
},
{\"text\": f\"Context: {state.answer_cot.relevant_context}\"},
{
\"text\": f\"Assertion: {state.answer_reformulation.declarative_answer}\"
},
{
\"text\": f\"Use this schema for your answer: {self.verification_cot_schema}. Be Factual.\"
},
],
}
]
response = self.model.generate_content(
messages,
generation_config={
\"response_mime_type\": \"application\/json\",
\"response_schema\": self.verification_cot_schema,
\"temperature\": 0.0,
},
)
verification_cot = VerificationChainOfThoughts(**json.loads(response.text))
return {\"verification_cot\": verification_cot}<\/pre>\nFull code in https:\/\/github.com\/CVxTz\/document_ai_agents<\/a><\/p>\nNotice how each node uses its own schema for structured output and its own prompt. This is possible due to the flexibility of both Gemini\u2019s API and LangGraph.<\/p>\n
Lets work through this code using the same example as above \u27a1\ufe0f
(Note: we are not using chain-of-thought on the first prompt so that the verification gets triggered for our\u00a0tests.)<\/em><\/p>\nContext<\/strong><\/p>\nThomas Jefferson (April 13 [O.S. April 2], 1743\u200a\u2014\u200aJuly 4, 1826) was an American statesman, planter, diplomat, lawyer, architect, philosopher, and Founding Father who served as the third president of the United States from 1801 to 1809.[6] He was the primary author of the Declaration of Independence. Following the American Revolutionary War and before becoming president in 1801, Jefferson was the nation\u2019s first U.S. secretary of state under George Washington and then the nation\u2019s second vice president under John Adams. Jefferson was a leading proponent of democracy, republicanism, and natural rights, and he produced formative documents and decisions at the state, national, and international levels. (Source: Wikipedia)<\/p><\/blockquote>\n
Question<\/strong><\/p>\nWhat year did davis jefferson die?<\/p><\/blockquote>\n
First node result (First\u00a0answer):<\/strong><\/p>\n\nrelevant_context<\/strong>=\u2019Thomas Jefferson (April 13 [O.S. April 2], 1743\u200a\u2014\u200aJuly 4, 1826) was an American statesman, planter, diplomat, lawyer, architect, philosopher, and Founding Father who served as the third president of the United States from 1801 to\u00a01809.\u2019<\/p><\/blockquote>\nanswer=\u20191826′<\/strong><\/p><\/blockquote>\nSecond node result (Answer Reformulation):<\/strong><\/p>\n\ndeclarative_answer<\/strong>=\u2019Davis Jefferson died in\u00a01826′<\/p><\/blockquote>\nThird node result (Verification):<\/strong><\/p>\n\nrationale<\/strong>=\u2019The context states that Thomas Jefferson died in 1826. The assertion states that Davis Jefferson died in 1826. The context does not mention Davis Jefferson, only Thomas Jefferson.\u2019<\/p><\/blockquote>\n\nentailment<\/strong>=\u2019No\u2019<\/p><\/blockquote>\nSo the verification step rejected<\/strong> (No entailment between the two<\/em>) the initial answer. We can now avoid returning a hallucination to the\u00a0user.<\/p>\nBonus Tip\u00a0: Use stronger\u00a0models<\/h4>\n
This tip is not always easy to apply due to budget or latency limitations but you should know that stronger LLMs are less prone to hallucination. So, if possible, go for a more powerful LLM for your most sensitive use cases. You can check a benchmark of hallucinations here: https:\/\/github.com\/vectara\/hallucination-leaderboard<\/a>. We can see that the top models in this benchmark (least hallucinations) also ranks at the top of conventional NLP leader\u00a0boards.<\/p>\n![\"\"](\"https:\/\/i0.wp.com\/cdn-images-1.medium.com\/max\/1024\/0%2A0lHK7eiqTV6ESKQQ.png?ssl=1\")
Source: https:\/\/github.com\/vectara\/hallucination-leaderboard<\/a> Source License: Apache\u00a02.0<\/figcaption><\/figure>\nConclusion<\/h3>\n
In this tutorial, we explored strategies to improve the reliability of LLM outputs by reducing the hallucination rate. The main recommendations include careful formatting and prompting to guide LLM calls and using a workflow based approach where Agents are designed to verify their own\u00a0answers.<\/p>\n
This involves multiple\u00a0steps:<\/p>\n
\n- Retrieving the exact context elements used by the LLM to generate the\u00a0answer.<\/li>\n
- Reformulating the answer for easier verification (In declarative form).<\/li>\n
- Instructing the LLM to check for consistency between the context and the reformulated answer.<\/li>\n<\/ol>\n
While all these tips can significantly improve accuracy, you should remember that no method is foolproof. There\u2019s always a risk of rejecting valid answers if the LLM is overly conservative during verification or missing real hallucination cases. Therefore, rigorous evaluation of your specific LLM workflows is still essential.<\/p>\n
Full code in https:\/\/github.com\/CVxTz\/document_ai_agents<\/a><\/p>\n![\"\"](\"https:\/\/medium.com\/_\/stat?event=post.clientViewed&referrerSource=full_rss&postId=f7ffd6eedcf2\")
<\/p>\n
\nAn Agentic Approach to Reducing LLM Hallucinations<\/a> was originally published in Towards Data Science<\/a> on Medium, where people are continuing the conversation by highlighting and responding to this story.<\/p>\n<\/div>\n
If you\u2019ve worked with LLMs, you know they can sometimes hallucinate. This means they generate text that\u2019s either nonsensical or contradicts the input data. It\u2019s a common issue that can hurts the reliability of LLM-powered applications.<\/p>\n
In this post, we\u2019ll explore a few simple techniques to reduce the likelihood of hallucinations. By following these tips, you can (hopefully) improve the accuracy of your AI applications.<\/p>\n
There are multiple types of hallucinations:<\/p>\n
- \n
- \nIntrinsic hallucinations<\/a>: the LLM\u2019s response contradicts the user-provided context. This is when the response is verifiably wrong withing the current\u00a0context.<\/li>\n
- \nExtrinsic hallucinations<\/a>: the LLM\u2019s response cannot be verified using the user-provided context. This is when the response may or may not be wrong but we have no way of confirming that using the current\u00a0context.<\/li>\n
- Incoherent hallucinations: the LLM\u2019s response does not answer the question or does not make sense. This is when the LLM is unable to follow the instructions.<\/li>\n<\/ul>\n
In this post, we will target all the types mentioned above.<\/p>\n
We will list out a set of tips and tricks that work in different ways in reducing hallucinations.<\/p>\n
Tip 1: Use Grounding<\/h4>\n
Grounding is using in-domain relevant additional context in the input of the LLM when asking it to do a task. This gives the LLM the information it needs to correctly answer the question and reduces the likelihood of a hallucination. This is one the reason we use Retrieval augmented generation (RAG).<\/p>\n
For example asking the LLM a math question OR asking it the same question while providing it with relevant sections of a math book will yield different results, with the second option being more likely to be\u00a0right.<\/p>\n
Here is an example of such implementation in one of my previous tutorials where I provide document-extracted context when asking a question:<\/p>\n
Build a Document AI pipeline for ANY type of PDF With Gemini<\/a><\/p>\n
Tip 2: Use structured outputs<\/h4>\n
Using structured outputs means forcing the LLM to output valid JSON or YAML text. This will allow you to reduce the useless ramblings and get \u201cstraight-to-the-point\u201d answers about what you need from the LLM. It also will help with the next tips as it makes the LLM responses easier to\u00a0verify.<\/p>\n
Here is how you can do this with Gemini\u2019s\u00a0API:<\/p>\n
import json
import google.generativeai as genai
from pydantic import BaseModel, Field
from document_ai_agents.schema_utils import prepare_schema_for_gemini
class Answer(BaseModel):
answer: str = Field(..., description=\"Your Answer.\")
model = genai.GenerativeModel(\"gemini-1.5-flash-002\")
answer_schema = prepare_schema_for_gemini(Answer)
question = \"List all the reasons why LLM hallucinate\"
context = (
\"LLM hallucination refers to the phenomenon where large language models generate plausible-sounding but\"
\" factually incorrect or nonsensical information. This can occur due to various factors, including biases\"
\" in the training data, the inherent limitations of the model's understanding of the real world, and the \"
\"model's tendency to prioritize fluency and coherence over accuracy.\"
)
messages = (
[context]
+ [
f\"Answer this question: {question}\",
]
+ [
f\"Use this schema for your answer: {answer_schema}\",
]
)
response = model.generate_content(
messages,
generation_config={
\"response_mime_type\": \"application\/json\",
\"response_schema\": answer_schema,
\"temperature\": 0.0,
},
)
response = Answer(**json.loads(response.text))
print(f\"{response.answer=}\")<\/pre>\nWhere \u201cprepare_schema_for_gemini\u201d is a utility function that prepares the schema to match Gemini\u2019s weird requirements. You can find its definition here:\u00a0code<\/a>.<\/p>\n
This code defines a Pydantic schema and sends this schema as part of the query in the field \u201cresponse_schema\u201d. This forces the LLM to follow this schema in its response and makes it easier to parse its\u00a0output.<\/p>\n
Tip 3: Use chain of thoughts and better prompting<\/h4>\n
Sometimes, giving the LLM the space to work out its response, before committing to a final answer, can help produce better quality responses. This technique is called Chain-of-thoughts and is widely used as it is effective and very easy to implement.<\/p>\n
We can also explicitly ask the LLM to answer with \u201cN\/A\u201d if it can\u2019t find enough context to produce a quality response. This will give it an easy way out instead of trying to respond to questions it has no answer\u00a0to.<\/p>\n
For example, lets look into this simple question and\u00a0context:<\/p>\n
Context<\/strong><\/p>\n
Thomas Jefferson (April 13 [O.S. April 2], 1743\u200a\u2014\u200aJuly 4, 1826) was an American statesman, planter, diplomat, lawyer, architect, philosopher, and Founding Father who served as the third president of the United States from 1801 to 1809.[6] He was the primary author of the Declaration of Independence. Following the American Revolutionary War and before becoming president in 1801, Jefferson was the nation\u2019s first U.S. secretary of state under George Washington and then the nation\u2019s second vice president under John Adams. Jefferson was a leading proponent of democracy, republicanism, and natural rights, and he produced formative documents and decisions at the state, national, and international levels. (Source: Wikipedia)<\/p><\/blockquote>\n
Question<\/strong><\/p>\n
What year did davis jefferson die?<\/p><\/blockquote>\n
A naive approach\u00a0yields:<\/p>\n
Response<\/strong><\/p>\n
answer=\u20191826′<\/p><\/blockquote>\n
Which is obviously false as Jefferson Davis is not even mentioned in the context at all. It was Thomas Jefferson that died in\u00a01826.<\/p>\n
If we change the schema of the response to use chain-of-thoughts to:<\/p>\n
class AnswerChainOfThoughts(BaseModel):
rationale: str = Field(
...,
description=\"Justification of your answer.\",
)
answer: str = Field(
..., description=\"Your Answer. Answer with 'N\/A' if answer is not found\"
)<\/pre>\nWe are also adding more details about what we expect as output when the question is not answerable using the context \u201cAnswer with \u2018N\/A\u2019 if answer is not\u00a0found\u201d<\/p>\n
With this new approach, we get the following rationale<\/strong> (remember, chain-of-thought):<\/p>\n
The provided text discusses Thomas Jefferson, not Jefferson Davis. No information about the death of Jefferson Davis is included.<\/p><\/blockquote>\n
And the final\u00a0answer<\/strong>:<\/p>\n
answer=\u2019N\/A\u2019<\/p><\/blockquote>\n
Great\u00a0! But can we use a more general approach to hallucination detection?<\/p>\n
We can, with\u00a0Agents!<\/p>\n
Tip 4: Use an Agentic\u00a0approach<\/h4>\n
We will build a simple agent that implements a three-step process:<\/p>\n
- \n
- The first step is to include the context and ask the question to the LLM in order to get the first candidate response and the relevant context that it had used for its\u00a0answer.<\/li>\n
- The second step is to reformulate the question and the first candidate response as a declarative statement.<\/li>\n
- The third step is to ask the LLM to verify whether or not the relevant context entails<\/strong> the candidate response. It is called \u201cSelf-verification\u201d: https:\/\/arxiv.org\/pdf\/2212.09561<\/a>\n<\/li>\n<\/ul>\n
In order to implement this, we define three nodes in LangGraph. The first node will ask the question while including the context, the second node will reformulate it using the LLM and the third node will check the entailment of the statement in relation to the input\u00a0context.<\/p>\n
The first node can be defined as\u00a0follows:<\/p>\n
def answer_question(self, state: DocumentQAState):
logger.info(f\"Responding to question '{state.question}'\")
assert (
state.pages_as_base64_jpeg_images or state.pages_as_text
), \"Input text or images\"
messages = (
[
{\"mime_type\": \"image\/jpeg\", \"data\": base64_jpeg}
for base64_jpeg in state.pages_as_base64_jpeg_images
]
+ state.pages_as_text
+ [
f\"Answer this question: {state.question}\",
]
+ [
f\"Use this schema for your answer: {self.answer_cot_schema}\",
]
)
response = self.model.generate_content(
messages,
generation_config={
\"response_mime_type\": \"application\/json\",
\"response_schema\": self.answer_cot_schema,
\"temperature\": 0.0,
},
)
answer_cot = AnswerChainOfThoughts(**json.loads(response.text))
return {\"answer_cot\": answer_cot}<\/pre>\nAnd the second one\u00a0as:<\/p>\n
def reformulate_answer(self, state: DocumentQAState):
logger.info(\"Reformulating answer\")
if state.answer_cot.answer == \"N\/A\":
return
messages = [
{
\"role\": \"user\",
\"parts\": [
{
\"text\": \"Reformulate this question and its answer as a single assertion.\"
},
{\"text\": f\"Question: {state.question}\"},
{\"text\": f\"Answer: {state.answer_cot.answer}\"},
]
+ [
{
\"text\": f\"Use this schema for your answer: {self.declarative_answer_schema}\"
}
],
}
]
response = self.model.generate_content(
messages,
generation_config={
\"response_mime_type\": \"application\/json\",
\"response_schema\": self.declarative_answer_schema,
\"temperature\": 0.0,
},
)
answer_reformulation = AnswerReformulation(**json.loads(response.text))
return {\"answer_reformulation\": answer_reformulation}<\/pre>\nThe third one\u00a0as:<\/p>\n
def verify_answer(self, state: DocumentQAState):
logger.info(f\"Verifying answer '{state.answer_cot.answer}'\")
if state.answer_cot.answer == \"N\/A\":
return
messages = [
{
\"role\": \"user\",
\"parts\": [
{
\"text\": \"Analyse the following context and the assertion and decide whether the context \"
\"entails the assertion or not.\"
},
{\"text\": f\"Context: {state.answer_cot.relevant_context}\"},
{
\"text\": f\"Assertion: {state.answer_reformulation.declarative_answer}\"
},
{
\"text\": f\"Use this schema for your answer: {self.verification_cot_schema}. Be Factual.\"
},
],
}
]
response = self.model.generate_content(
messages,
generation_config={
\"response_mime_type\": \"application\/json\",
\"response_schema\": self.verification_cot_schema,
\"temperature\": 0.0,
},
)
verification_cot = VerificationChainOfThoughts(**json.loads(response.text))
return {\"verification_cot\": verification_cot}<\/pre>\nFull code in https:\/\/github.com\/CVxTz\/document_ai_agents<\/a><\/p>\n
Notice how each node uses its own schema for structured output and its own prompt. This is possible due to the flexibility of both Gemini\u2019s API and LangGraph.<\/p>\n
Lets work through this code using the same example as above \u27a1\ufe0f
(Note: we are not using chain-of-thought on the first prompt so that the verification gets triggered for our\u00a0tests.)<\/em><\/p>\nContext<\/strong><\/p>\n
Thomas Jefferson (April 13 [O.S. April 2], 1743\u200a\u2014\u200aJuly 4, 1826) was an American statesman, planter, diplomat, lawyer, architect, philosopher, and Founding Father who served as the third president of the United States from 1801 to 1809.[6] He was the primary author of the Declaration of Independence. Following the American Revolutionary War and before becoming president in 1801, Jefferson was the nation\u2019s first U.S. secretary of state under George Washington and then the nation\u2019s second vice president under John Adams. Jefferson was a leading proponent of democracy, republicanism, and natural rights, and he produced formative documents and decisions at the state, national, and international levels. (Source: Wikipedia)<\/p><\/blockquote>\n
Question<\/strong><\/p>\n
What year did davis jefferson die?<\/p><\/blockquote>\n
First node result (First\u00a0answer):<\/strong><\/p>\n
\nrelevant_context<\/strong>=\u2019Thomas Jefferson (April 13 [O.S. April 2], 1743\u200a\u2014\u200aJuly 4, 1826) was an American statesman, planter, diplomat, lawyer, architect, philosopher, and Founding Father who served as the third president of the United States from 1801 to\u00a01809.\u2019<\/p><\/blockquote>\n
answer=\u20191826′<\/strong><\/p><\/blockquote>\n
Second node result (Answer Reformulation):<\/strong><\/p>\n
\ndeclarative_answer<\/strong>=\u2019Davis Jefferson died in\u00a01826′<\/p><\/blockquote>\n
Third node result (Verification):<\/strong><\/p>\n
\nrationale<\/strong>=\u2019The context states that Thomas Jefferson died in 1826. The assertion states that Davis Jefferson died in 1826. The context does not mention Davis Jefferson, only Thomas Jefferson.\u2019<\/p><\/blockquote>\n
\nentailment<\/strong>=\u2019No\u2019<\/p><\/blockquote>\n
So the verification step rejected<\/strong> (No entailment between the two<\/em>) the initial answer. We can now avoid returning a hallucination to the\u00a0user.<\/p>\n
Bonus Tip\u00a0: Use stronger\u00a0models<\/h4>\n
This tip is not always easy to apply due to budget or latency limitations but you should know that stronger LLMs are less prone to hallucination. So, if possible, go for a more powerful LLM for your most sensitive use cases. You can check a benchmark of hallucinations here: https:\/\/github.com\/vectara\/hallucination-leaderboard<\/a>. We can see that the top models in this benchmark (least hallucinations) also ranks at the top of conventional NLP leader\u00a0boards.<\/p>\n
Source: https:\/\/github.com\/vectara\/hallucination-leaderboard<\/a> Source License: Apache\u00a02.0<\/figcaption><\/figure>\n Conclusion<\/h3>\n
In this tutorial, we explored strategies to improve the reliability of LLM outputs by reducing the hallucination rate. The main recommendations include careful formatting and prompting to guide LLM calls and using a workflow based approach where Agents are designed to verify their own\u00a0answers.<\/p>\n
This involves multiple\u00a0steps:<\/p>\n
- \n
- Retrieving the exact context elements used by the LLM to generate the\u00a0answer.<\/li>\n
- Reformulating the answer for easier verification (In declarative form).<\/li>\n
- Instructing the LLM to check for consistency between the context and the reformulated answer.<\/li>\n<\/ol>\n
While all these tips can significantly improve accuracy, you should remember that no method is foolproof. There\u2019s always a risk of rejecting valid answers if the LLM is overly conservative during verification or missing real hallucination cases. Therefore, rigorous evaluation of your specific LLM workflows is still essential.<\/p>\n
Full code in https:\/\/github.com\/CVxTz\/document_ai_agents<\/a><\/p>\n
\nAn Agentic Approach to Reducing LLM Hallucinations<\/a> was originally published in Towards Data Science<\/a> on Medium, where people are continuing the conversation by highlighting and responding to this story.<\/p>\n<\/div>\n
- \nExtrinsic hallucinations<\/a>: the LLM\u2019s response cannot be verified using the user-provided context. This is when the response may or may not be wrong but we have no way of confirming that using the current\u00a0context.<\/li>\n