Introduction<\/strong><\/h2>\nMany of us might have tried to build a RAG application and noticed it falls significantly short of addressing real-life needs. Why is that? It\u2019s because many real-world problems require multiple steps of information retrieval and reasoning. We need our agent to perform those as humans normally do, yet most RAG applications fall short of this.<\/p>\n
This article explores how to supercharge your RAG application by making its data retrieval and reasoning process similar to how a human would, under a multi-agent framework. The framework presented here is based on the\u00a0Self-RAG strategy<\/a>\u00a0but has been significantly modified to enhance its capabilities. Prior knowledge of the original strategy is not necessary for reading this article.<\/p>\nReal-life Case<\/strong><\/h2>\nConsider this: I was going to fly from Delhi to Munich (let\u2019s assume I am taking the flight from an EU airline), but I was denied boarding somehow. Now I want to know what the compensation should be.<\/p>\n
These two webpages contain relevant information, I go ahead adding them to my vector store, trying to have my agent answer this for me by retrieving the right information.<\/p>\n
\n- Flight compensation policy<\/a><\/li>\n
- \nDistance between Munich and other major cities in the world<\/a>\u00a0(note: it does not reflect the distance between airports, but it serves the purpose for our demonstration)<\/li>\n<\/ul>\n
Now, I pass this question to the vector store:\u00a0\u201chow much can I receive if I am denied boarding, for flights from Delhi to Munich?\u201d<\/em><\/strong>.<\/p>\n\u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013<br>Overview of US Flight Compensation Policies To get compensation for delayed flights, you should contact your airline via their customer service or go to the customer service desk. At the same time, you should bear in mind that you will only receive compensation if the delay is not weather-related and is within the carrier`s control. According to the US Department of Transportation, US airlines are not required to compensate you if a flight is cancelled or delayed. You can be compensated if you are bumped or moved from an overbooked flight. If your provider cancels your flight less than two weeks before departure and you decide to cancel your trip entirely, you can receive a refund of both pre-paid baggage fees and your plane ticket. There will be no refund if you choose to continue your journey. In the case of a delayed flight, the airline will rebook you on a different flight. According to federal law, you will not be provided with money or other compensation. Comparative Analysis of EU vs. US Flight Compensation Policies<br>\u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013<br><strong>(AUTHOR-ADDED NOTE: IMPORTANT, PAY ATTENTION TO THIS)<\/strong><br>Short-distance flight delays \u2013 if it is up to 1,500 km, you are due 250 Euro compensation.<br>Medium distance flight delays \u2013 for all the flights between 1,500 and 3,500 km, the compensation should be 400 Euro.<br>Long-distance flight delays \u2013 if it is over 3,500 km, you are due 600 Euro compensation. To receive this kind of compensation, the following conditions must be met; Your flight starts in a non-EU member state or in an EU member state and finishes in an EU member state and is organised by an EU airline. Your flight reaches the final destination with a delay that exceeds three hours. There is no force majeure.<br>\u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013<br>Compensation policies in the EU and US are not the same, which implies that it is worth knowing more about them. While you can always count on Skycop flight cancellation compensation, you should still get acquainted with the information below.<br>\u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013<br>Compensation for flight regulations EU: The EU does regulate flight delay compensation, which is known as EU261. US: According to the US Department of Transportation, every airline has its own policies about what should be done for delayed passengers. Compensation for flight delays EU: Just like in the United States, compensation is not provided when the flight is delayed due to uncontrollable reasons. However, there is a clear approach to compensation calculation based on distance. For example, if your flight was up to 1,500 km, you can receive 250 euros. US: There are no federal requirements. That is why every airline sets its own limits for compensation in terms of length. However, it is usually set at three hours. Overbooking EU: In the EU, they call for volunteers if the flight is overbooked. These people are entitled to a choice of: Re-routing to their final destination at the earliest opportunity. Refund of their ticket cost within a week if not travelling. Re-routing at a later date at the person`s convenience.<\/code><\/pre>\nUnfortunately, they contain only generic flight compensation policies, without telling me how much I can expect when denied boarding from Delhi to Munich specifically. If the RAG agent takes these as the sole context, it can only provide a generic answer about flight compensation policy, without giving the answer we want.<\/p>\n
However, while the documents are not immediately useful, there is an important insight contained in the 2nd piece of context:\u00a0compensation varies according to flight distance<\/em><\/strong>. If the RAG agent thinks more like human, it should follow these steps to provide an answer:<\/p>\n\n- Based on the retrieved context, reason that compensation varies with flight distance<\/li>\n
- Next, retrieve the flight distance between Delhi and Munich<\/li>\n
- Given the distance (which is around 5900km), classify the flight as a long-distance one<\/li>\n
- Combined with the previously retrieved context, figure out I am due 600 EUR, assuming other conditions are fulfilled<\/li>\n<\/ol>\n
This example demonstrates how a simple RAG, in which a single retrieval is made, fall short for several reasons:<\/p>\n
\n- \nComplex Queries<\/strong>: Users often have questions that a simple search can\u2019t fully address. For example, \u201cWhat\u2019s the best smartphone for gaming under $500?\u201d requires consideration of multiple factors like performance, price, and features, which a single retrieval step might miss.<\/li>\n
- \nDeep Information<\/strong>: Some information lies across documents. For example, research papers, medical records, or legal documents often include references that need to be made sense of, before one can fully understand the content of a given article. Multiple retrieval steps help dig deeper into the content.<\/li>\n<\/ol>\n
Multiple retrievals supplemented with human-like reasoning allow for a more nuanced, comprehensive, and accurate response, adapting to the complexity and depth of user queries.<\/p>\n
Multi-Agent Self-RAG<\/h2>\n
Here I explain the reasoning process behind this strategy, afterwards I will provide the code to show you how to achieve this!<\/p>\n
\nNote: For readers interested in knowing how my approach differs from the\u00a0original Self-RAG<\/a>, I will describe the discrepancies in quotation boxes like this. But general readers who are unfamiliar with the original Self-RAG can skip them.<\/p>\n<\/blockquote>\nIn the below graphs, each circle represents a step (aka\u00a0Node<\/strong>), which is performed by a dedicated agent working on the specific problem. We orchestrate them to form a multi-agent RAG application.<\/p>\n1st iteration: Simple RAG<\/h3>\n![\"\"](\"https:\/\/i0.wp.com\/towardsdatascience.com\/wp-content\/uploads\/2025\/02\/1_7cjs7z09W_HEySmLc0-whw-1.png?resize=895%2C226&ssl=1\")
A simple RAG chain<\/figcaption><\/figure>\nThis is just the vanilla RAG approach I described in \u201cReal-life Case\u201d, represented as a graph. After\u00a0Retrieve documents<\/code>, the\u00a0new_documents<\/code>\u00a0will be used as input for\u00a0Generate Answer<\/code>. Nothing special, but it serves as our starting point.<\/p>\n2nd iteration: Digest documents with \u201cGrade documents\u201d<\/h3>\n![\"\"](\"https:\/\/i0.wp.com\/towardsdatascience.com\/wp-content\/uploads\/2025\/02\/1_S7emqxgrROrusM2R4K0N2w-1024x261.png?resize=1024%2C261&ssl=1\")
Reasoning like human do<\/figcaption><\/figure>\nRemember I said in the \u201cReal-life Case\u201d section, that as a next step, the agent should\u00a0\u201creason that compensation varies with flight distance\u201d<\/em><\/strong>? The\u00a0Grade documents<\/code>\u00a0step is exactly for this purpose.<\/p>\nGiven the\u00a0new_documents<\/code>, the agent will try to output two items:<\/p>\n\n- \n
useful_documents<\/code>: Comparing the question asked, it determines if the documents are useful, and retain a memory for those deemed useful for future reference. As an example, since our question does not concern compensation policies for US, documents describing those are discarded, leaving only those for EU<\/li>\n- \n
hypothesis<\/code>: Based on the documents, the agent forms a hypothesis about how the question can be answered, that is, flight distance needs to be identified<\/li>\n<\/ol>\nNotice how the above reasoning resembles human thinking! But still, while these outputs are useful, we need to instruct the agent to use them as input for performing the next document retrieval. Without this, the answer provided in\u00a0Generate answer<\/code>\u00a0is still not useful.<\/p>\n\nuseful_documents<\/code>\u00a0are appended for each document retrieval loop, instead of being overwritten, to keep a memory of documents that are previously deemed useful.\u00a0hypothesis<\/code>\u00a0is formed from\u00a0useful_documents<\/code>\u00a0and\u00a0new_documents<\/code>\u00a0to provide an \u201cabstract reasoning\u201d to inform how query is to be transformed subsequently.<\/p>\nThe\u00a0hypothesis<\/code>\u00a0is especially useful when no useful documents can be identified initially, as the agent can still form hypothesis from documents not immediately deemed as useful \/ only bearing indirect relationship to the question at hand, for informing what questions to ask next<\/p>\n<\/blockquote>\n3rd iteration: Brainstorm new questions to ask<\/h3>\n![\"\"](\"https:\/\/i0.wp.com\/towardsdatascience.com\/wp-content\/uploads\/2025\/02\/1_xI3fGpV0M6BfLPIZDtD_2Q-1-1024x355.png?resize=1024%2C355&ssl=1\")
Suggest questions for additional information retrieval<\/figcaption><\/figure>\nWe have the agent reflect upon whether the answer is useful and grounded in context. If not, it should proceed to\u00a0Transform query<\/code>\u00a0to ask further questions.<\/p>\nThe output\u00a0new_queries<\/code>\u00a0will be a list of new questions that the agent consider useful for obtaining extra information. Given the\u00a0useful_documents<\/code>\u00a0(compensation policies for EU), and\u00a0hypothesis<\/code>\u00a0(need to identify flight distance between Delhi and Munich), it asks questions like\u00a0\u201cWhat is the distance between Delhi and Munich?\u201d<\/strong><\/p>\nNow we are ready to use the\u00a0new_queries<\/code>\u00a0for further retrieval!<\/p>\n\nThe\u00a0transform_query<\/code>\u00a0node will use\u00a0useful_documents<\/code>\u00a0(which are accumulated per iteration, instead of being overwritten) and\u00a0hypothesis<\/code>\u00a0as input for providing the agent directions to ask new questions.<\/p>\nThe new questions will be a list of questions (instead of a single question) separated from the original\u00a0question<\/code>, so that the original question is kept in state, otherwise the agent could lose track of the original question after multiple iterations.<\/p>\n<\/blockquote>\nFinal iteration: Further retrieval with new questions<\/h3>\n![\"\"](\"https:\/\/i0.wp.com\/towardsdatascience.com\/wp-content\/uploads\/2025\/02\/1_g2kgeEqpTlWhHntGo7ip6Q-1-1024x439.png?resize=1024%2C439&ssl=1\")
Issuing new queries to retrieve extra documents<\/figcaption><\/figure>\nThe output\u00a0new_queries<\/code>\u00a0from\u00a0Transform query<\/code>\u00a0will be passed to the\u00a0Retrieve documents<\/code>\u00a0step, forming a retrieval loop.<\/p>\nSince the question\u00a0\u201cWhat is the distance between Delhi and Munich?\u201d<\/em><\/strong>\u00a0<\/em>is asked, we can expect the flight distance is then retrieved as\u00a0new_documents<\/code>, and subsequently graded as\u00a0useful_documents<\/code>, further used as an input for\u00a0Generate answer<\/code>.<\/p>\n\nThe\u00a0<em>grade_documents<\/em><\/code>\u00a0node will compare the documents against both the original\u00a0<em>question<\/em><\/code>\u00a0and\u00a0<em>new_questions<\/em><\/code>\u00a0list, so that documents that are considered useful for\u00a0<em>new_questions<\/em><\/code>, even if not so for the original\u00a0<em>question<\/em><\/code>, are kept.<\/p>\nThis is because those documents might help answer the original\u00a0<em>question<\/em><\/code>\u00a0indirectly, by being relevant to\u00a0<em>new_questions<\/em><\/code>\u00a0(like \u201c<\/em>What is the distance between Delhi and Munich?\u201d)<\/p>\n<\/blockquote>\n![\"\"](\"https:\/\/i0.wp.com\/towardsdatascience.com\/wp-content\/uploads\/2025\/02\/1_g2kgeEqpTlWhHntGo7ip6Q-3-1024x439.png?resize=1024%2C439&ssl=1\")
Final answer!<\/figcaption><\/figure>\nEquipped with this new context about flight distance, the agent is now ready to provide the right answer:\u00a0600 EUR<\/strong>!<\/p>\nNext, let us now dive into the code to see how this multi-agent RAG application is created.<\/p>\n
Implementation<\/h2>\n
The source code can be found\u00a0here<\/a>. Our multi-agent RAG application involves iterations and loops, and LangGraph is a great library for building such complex multi-agent application. If you are not familiar with LangGraph, you are strongly suggested to have a look at\u00a0LangGraph\u2019s Quickstart guide<\/a>\u00a0to understand more about it!<\/p>\nTo keep this article concise, I will focus on the key code snippets only.<\/p>\n
Important note<\/strong>: I am using OpenRouter as the\u00a0Llm<\/a>\u00a0interface, but the code can be easily adapted for other LLM interfaces. Also, while in my code I am using Claude 3.5 Sonnet as model, you can use any LLM as long as it support\u00a0tools<\/code>\u00a0as parameter (check this list\u00a0here<\/a>), so you can also run this with other models, like DeepSeek V3 and OpenAI o1!<\/p>\nState definition<\/h3>\n
In the previous section, I have defined various elements e.g.\u00a0new_documents<\/code>,\u00a0hypothesis<\/code>\u00a0that are to be passed to each step (aka\u00a0Nodes<\/strong>), in LangGraph\u2019s terminology these elements are called\u00a0State<\/strong>.<\/p>\nWe define the State formally with the following snippet.<\/p>\n
from typing import List, Annotated<br>from typing_extensions import TypedDict<br><br>def append_to_list(original: list, new: list) -> list:<br> original.append(new)<br> return original<br><br>def combine_list(original: list, new: list) -> list:<br> return original + new<br><br>class GraphState(TypedDict):<br> \"\"\"<br> Represents the state of our graph.<br><br> Attributes:<br> question: question<br> generation: LLM generation<br> new_documents: newly retrieved documents for the current iteration<br> useful_documents: documents that are considered useful<br> graded_documents: documents that have been graded<br> new_queries: newly generated questions<br> hypothesis: hypothesis<br> \"\"\"<br><br> question: str<br> generation: str<br> new_documents: List[str]<br> useful_documents: Annotated[List[str], combine_list]<br> graded_documents: List[str]<br> new_queries: Annotated[List[str], append_to_list]<br> hypothesis: str<\/code><\/pre>\nGraph definition<\/h3>\n
This is where we combine the different steps to form a \u201cGraph<\/strong>\u201d, which is a representation of our multi-agent application. The definitions of various steps (e.g.\u00a0grade_documents<\/code>) are represented by their respective functions.<\/p>\nfrom langgraph.graph import END, StateGraph, START<br>from langgraph.checkpoint.memory import MemorySaver<br>from IPython.display import Image, display<br><br>workflow = StateGraph(GraphState)<br><br># Define the nodes<br>workflow.add_node(\"retrieve\", retrieve) # retrieve<br>workflow.add_node(\"grade_documents\", grade_documents) # grade documents<br>workflow.add_node(\"generate\", generate) # generatae<br>workflow.add_node(\"transform_query\", transform_query) # transform_query<br><br># Build graph<br>workflow.add_edge(START, \"retrieve\")<br>workflow.add_edge(\"retrieve\", \"grade_documents\")<br>workflow.add_conditional_edges(<br> \"grade_documents\",<br> decide_to_generate,<br> {<br> \"transform_query\": \"transform_query\",<br> \"generate\": \"generate\",<br> },<br>)<br>workflow.add_edge(\"transform_query\", \"retrieve\")<br>workflow.add_conditional_edges(<br> \"generate\",<br> grade_generation_v_documents_and_question,<br> {<br> \"useful\": END,<br> \"not supported\": \"transform_query\",<br> \"not useful\": \"transform_query\",<br> },<br>)<br><br># Compile<br>memory = MemorySaver()<br>app = workflow.compile(checkpointer=memory)<br>display(Image(app.get_graph(xray=True).draw_mermaid_png()))<\/code><\/pre>\nRunning the above code, you should see this graphical representation of our RAG application. Notice how it is essentially equivalent to the graph I have shown in the final iteration of \u201cEnhanced Self-RAG Strategy\u201d!<\/p>\n![\"\"](\"https:\/\/i0.wp.com\/towardsdatascience.com\/wp-content\/uploads\/2025\/02\/1_bRGuqH1Zm577pt6nDJIPWQ.png?resize=515%2C557&ssl=1\")
Visualizing the multi-agent RAG graph<\/figcaption><\/figure>\n\nAfter\u00a0generate<\/code>, if the answer is considered \u201cnot supported\u201d, the agent will proceed to\u00a0transform_query<\/code>\u00a0intead of to\u00a0generate<\/code>\u00a0again, so that the agent will look for additional information rather than trying to regenerate answers based on existing context, which might not suffice for providing a \u201csupported\u201d answer<\/p>\n<\/blockquote>\nNow we are ready to put the multi-agent application to test! With the below code snippet, we ask this question\u00a0how much can I receive if I am denied boarding, for flights from Delhi to Munich?<\/code><\/p>\nfrom pprint import pprint<br>config = {\"configurable\": {\"thread_id\": str(uuid4())}}<br><br># Run<br>inputs = {<br> \"question\": \"how much can I receive if I am denied boarding, for flights from Delhi to Munich?\",<br> }<br>for output in app.stream(inputs, config):<br> for key, value in output.items():<br> # Node<br> pprint(f\"Node '{key}':\")<br> # Optional: print full state at each node<br> # print(app.get_state(config).values)<br> pprint(\"n---n\")<br><br># Final generation<br>pprint(value[\"generation\"])<\/code><\/pre>\nWhile output might vary (sometimes the application provides the answer without any iterations, because it \u201cguessed\u201d the distance between Delhi and Munich), it should look something like this, which shows the application went through multiple rounds of data retrieval for RAG.<\/p>\n
---RETRIEVE---<br>\"Node 'retrieve':\"<br>'n---n'<br>---CHECK DOCUMENT RELEVANCE TO QUESTION---<br>---GRADE: DOCUMENT NOT RELEVANT---<br>---GRADE: DOCUMENT RELEVANT---<br>---GRADE: DOCUMENT NOT RELEVANT---<br>---GRADE: DOCUMENT NOT RELEVANT---<br>---ASSESS GRADED DOCUMENTS---<br>---DECISION: GENERATE---<br>\"Node 'grade_documents':\"<br>'n---n'<br>---GENERATE---<br>---CHECK HALLUCINATIONS---<br>'---DECISION: GENERATION IS NOT GROUNDED IN DOCUMENTS, RE-TRY---'<br>\"Node 'generate':\"<br>'n---n'<br>---TRANSFORM QUERY---<br>\"Node 'transform_query':\"<br>'n---n'<br>---RETRIEVE---<br>\"Node 'retrieve':\"<br>'n---n'<br>---CHECK DOCUMENT RELEVANCE TO QUESTION---<br>---GRADE: DOCUMENT NOT RELEVANT---<br>---GRADE: DOCUMENT NOT RELEVANT---<br>---GRADE: DOCUMENT RELEVANT---<br>---GRADE: DOCUMENT NOT RELEVANT---<br>---GRADE: DOCUMENT NOT RELEVANT---<br>---GRADE: DOCUMENT NOT RELEVANT---<br>---GRADE: DOCUMENT NOT RELEVANT---<br>---ASSESS GRADED DOCUMENTS---<br>---DECISION: GENERATE---<br>\"Node 'grade_documents':\"<br>'n---n'<br>---GENERATE---<br>---CHECK HALLUCINATIONS---<br>---DECISION: GENERATION IS GROUNDED IN DOCUMENTS---<br>---GRADE GENERATION vs QUESTION---<br>---DECISION: GENERATION ADDRESSES QUESTION---<br>\"Node 'generate':\"<br>'n---n'<br>('Based on the context provided, the flight distance from Munich to Delhi is '<br> '5,931 km, which falls into the long-distance category (over 3,500 km). '<br> 'Therefore, if you are denied boarding on a flight from Delhi to Munich '<br> 'operated by an EU airline, you would be eligible for 600 Euro compensation, '<br> 'provided that:n'<br> '1. The flight is operated by an EU airlinen'<br> '2. There is no force majeuren'<br> '3. Other applicable conditions are metn'<br> 'n'<br> \"However, it's important to note that this compensation amount is only valid \"<br> 'if all the required conditions are met as specified in the regulations.')<\/code><\/pre>\nAnd the final answer is what we aimed for!<\/p>\n
Based on the context provided, the flight distance from Munich to Delhi is<br>5,931 km, which falls into the long-distance category (over 3,500 km).<br>Therefore, if you are denied boarding on a flight from Delhi to Munich<br>operated by an EU airline, you would be eligible for 600 Euro compensation,<br>provided that:<br>1. The flight is operated by an EU airline<br>2. There is no force majeure<br>3. Other applicable conditions are met<br><br>However, it's important to note that this compensation amount is only valid<br>if all the required conditions are met as specified in the regulations.<\/code><\/pre>\nInspecting the State, we see how the\u00a0hypothesis<\/code>\u00a0and\u00a0new_queries<\/code>\u00a0enhance the effectiveness of our multi-agent RAG application by mimicking human thinking process.<\/p>\nHypothesis<\/strong><\/p>\nprint(app.get_state(config).values.get('hypothesis',\"\"))<\/code><\/pre>\n--- Output ---<br>To answer this question accurately, I need to determine:<br><br>1. Is this flight operated by an EU airline? (Since Delhi is non-EU and Munich is EU)<br>2. What is the flight distance between Delhi and Munich? (To determine compensation amount)<br>3. Are we dealing with a denied boarding situation due to overbooking? (As opposed to delay\/cancellation)<br><br>From the context, I can find information about compensation amounts based on distance, but I need to verify:<br>- If the flight meets EU compensation eligibility criteria<br>- The exact distance between Delhi and Munich to determine which compensation tier applies (250\u20ac, 400\u20ac, or 600\u20ac)<br>- If denied boarding compensation follows the same amounts as delay compensation<br><br>The context doesn't explicitly state compensation amounts specifically for denied boarding, though it mentions overbooking situations in the EU require offering volunteers re-routing or refund options.<br><br>Would you like me to proceed with the information available, or would you need additional context about denied boarding compensation specifically?<\/code><\/pre>\nNew Queries<\/strong><\/p>\nfor questions_batch in app.get_state(config).values.get('new_queries',\"\"):<br> for q in questions_batch:<br> print(q)<\/code><\/pre>\n--- Output ---<br>What is the flight distance between Delhi and Munich?<br>Does EU denied boarding compensation follow the same amounts as flight delay compensation?<br>Are there specific compensation rules for denied boarding versus flight delays for flights from non-EU to EU destinations?<br>What are the compensation rules when flying with non-EU airlines from Delhi to Munich?<br>What are the specific conditions that qualify as denied boarding under EU regulations?<\/code><\/pre>\nConclusion<\/h2>\n
Simple RAG, while easy to build, might fall short in tackling real-life questions. By incorporating human thinking process into a multi-agent RAG framework, we are making RAG applications much more practical.<\/p>\n
*Unless otherwise noted, all images are by the author<\/em><\/p>\n
\nThe post Supercharge Your RAG with Multi-Agent Self-RAG<\/a> appeared first on Towards Data Science<\/a>.<\/p>\n<\/div>\n
Real-life Case<\/strong><\/h2>\nConsider this: I was going to fly from Delhi to Munich (let\u2019s assume I am taking the flight from an EU airline), but I was denied boarding somehow. Now I want to know what the compensation should be.<\/p>\n
These two webpages contain relevant information, I go ahead adding them to my vector store, trying to have my agent answer this for me by retrieving the right information.<\/p>\n
\n- Flight compensation policy<\/a><\/li>\n
- \nDistance between Munich and other major cities in the world<\/a>\u00a0(note: it does not reflect the distance between airports, but it serves the purpose for our demonstration)<\/li>\n<\/ul>\n
Now, I pass this question to the vector store:\u00a0\u201chow much can I receive if I am denied boarding, for flights from Delhi to Munich?\u201d<\/em><\/strong>.<\/p>\n\u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013<br>Overview of US Flight Compensation Policies To get compensation for delayed flights, you should contact your airline via their customer service or go to the customer service desk. At the same time, you should bear in mind that you will only receive compensation if the delay is not weather-related and is within the carrier`s control. According to the US Department of Transportation, US airlines are not required to compensate you if a flight is cancelled or delayed. You can be compensated if you are bumped or moved from an overbooked flight. If your provider cancels your flight less than two weeks before departure and you decide to cancel your trip entirely, you can receive a refund of both pre-paid baggage fees and your plane ticket. There will be no refund if you choose to continue your journey. In the case of a delayed flight, the airline will rebook you on a different flight. According to federal law, you will not be provided with money or other compensation. Comparative Analysis of EU vs. US Flight Compensation Policies<br>\u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013<br><strong>(AUTHOR-ADDED NOTE: IMPORTANT, PAY ATTENTION TO THIS)<\/strong><br>Short-distance flight delays \u2013 if it is up to 1,500 km, you are due 250 Euro compensation.<br>Medium distance flight delays \u2013 for all the flights between 1,500 and 3,500 km, the compensation should be 400 Euro.<br>Long-distance flight delays \u2013 if it is over 3,500 km, you are due 600 Euro compensation. To receive this kind of compensation, the following conditions must be met; Your flight starts in a non-EU member state or in an EU member state and finishes in an EU member state and is organised by an EU airline. Your flight reaches the final destination with a delay that exceeds three hours. There is no force majeure.<br>\u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013<br>Compensation policies in the EU and US are not the same, which implies that it is worth knowing more about them. While you can always count on Skycop flight cancellation compensation, you should still get acquainted with the information below.<br>\u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013<br>Compensation for flight regulations EU: The EU does regulate flight delay compensation, which is known as EU261. US: According to the US Department of Transportation, every airline has its own policies about what should be done for delayed passengers. Compensation for flight delays EU: Just like in the United States, compensation is not provided when the flight is delayed due to uncontrollable reasons. However, there is a clear approach to compensation calculation based on distance. For example, if your flight was up to 1,500 km, you can receive 250 euros. US: There are no federal requirements. That is why every airline sets its own limits for compensation in terms of length. However, it is usually set at three hours. Overbooking EU: In the EU, they call for volunteers if the flight is overbooked. These people are entitled to a choice of: Re-routing to their final destination at the earliest opportunity. Refund of their ticket cost within a week if not travelling. Re-routing at a later date at the person`s convenience.<\/code><\/pre>\nUnfortunately, they contain only generic flight compensation policies, without telling me how much I can expect when denied boarding from Delhi to Munich specifically. If the RAG agent takes these as the sole context, it can only provide a generic answer about flight compensation policy, without giving the answer we want.<\/p>\n
However, while the documents are not immediately useful, there is an important insight contained in the 2nd piece of context:\u00a0compensation varies according to flight distance<\/em><\/strong>. If the RAG agent thinks more like human, it should follow these steps to provide an answer:<\/p>\n\n- Based on the retrieved context, reason that compensation varies with flight distance<\/li>\n
- Next, retrieve the flight distance between Delhi and Munich<\/li>\n
- Given the distance (which is around 5900km), classify the flight as a long-distance one<\/li>\n
- Combined with the previously retrieved context, figure out I am due 600 EUR, assuming other conditions are fulfilled<\/li>\n<\/ol>\n
This example demonstrates how a simple RAG, in which a single retrieval is made, fall short for several reasons:<\/p>\n
\n- \nComplex Queries<\/strong>: Users often have questions that a simple search can\u2019t fully address. For example, \u201cWhat\u2019s the best smartphone for gaming under $500?\u201d requires consideration of multiple factors like performance, price, and features, which a single retrieval step might miss.<\/li>\n
- \nDeep Information<\/strong>: Some information lies across documents. For example, research papers, medical records, or legal documents often include references that need to be made sense of, before one can fully understand the content of a given article. Multiple retrieval steps help dig deeper into the content.<\/li>\n<\/ol>\n
Multiple retrievals supplemented with human-like reasoning allow for a more nuanced, comprehensive, and accurate response, adapting to the complexity and depth of user queries.<\/p>\n
Multi-Agent Self-RAG<\/h2>\n
Here I explain the reasoning process behind this strategy, afterwards I will provide the code to show you how to achieve this!<\/p>\n
\nNote: For readers interested in knowing how my approach differs from the\u00a0original Self-RAG<\/a>, I will describe the discrepancies in quotation boxes like this. But general readers who are unfamiliar with the original Self-RAG can skip them.<\/p>\n<\/blockquote>\nIn the below graphs, each circle represents a step (aka\u00a0Node<\/strong>), which is performed by a dedicated agent working on the specific problem. We orchestrate them to form a multi-agent RAG application.<\/p>\n1st iteration: Simple RAG<\/h3>\nA simple RAG chain<\/figcaption><\/figure>\nThis is just the vanilla RAG approach I described in \u201cReal-life Case\u201d, represented as a graph. After\u00a0Retrieve documents<\/code>, the\u00a0new_documents<\/code>\u00a0will be used as input for\u00a0Generate Answer<\/code>. Nothing special, but it serves as our starting point.<\/p>\n2nd iteration: Digest documents with \u201cGrade documents\u201d<\/h3>\nReasoning like human do<\/figcaption><\/figure>\nRemember I said in the \u201cReal-life Case\u201d section, that as a next step, the agent should\u00a0\u201creason that compensation varies with flight distance\u201d<\/em><\/strong>? The\u00a0Grade documents<\/code>\u00a0step is exactly for this purpose.<\/p>\nGiven the\u00a0new_documents<\/code>, the agent will try to output two items:<\/p>\n\n- \n
useful_documents<\/code>: Comparing the question asked, it determines if the documents are useful, and retain a memory for those deemed useful for future reference. As an example, since our question does not concern compensation policies for US, documents describing those are discarded, leaving only those for EU<\/li>\n- \n
hypothesis<\/code>: Based on the documents, the agent forms a hypothesis about how the question can be answered, that is, flight distance needs to be identified<\/li>\n<\/ol>\nNotice how the above reasoning resembles human thinking! But still, while these outputs are useful, we need to instruct the agent to use them as input for performing the next document retrieval. Without this, the answer provided in\u00a0Generate answer<\/code>\u00a0is still not useful.<\/p>\n\nuseful_documents<\/code>\u00a0are appended for each document retrieval loop, instead of being overwritten, to keep a memory of documents that are previously deemed useful.\u00a0hypothesis<\/code>\u00a0is formed from\u00a0useful_documents<\/code>\u00a0and\u00a0new_documents<\/code>\u00a0to provide an \u201cabstract reasoning\u201d to inform how query is to be transformed subsequently.<\/p>\nThe\u00a0hypothesis<\/code>\u00a0is especially useful when no useful documents can be identified initially, as the agent can still form hypothesis from documents not immediately deemed as useful \/ only bearing indirect relationship to the question at hand, for informing what questions to ask next<\/p>\n<\/blockquote>\n3rd iteration: Brainstorm new questions to ask<\/h3>\nSuggest questions for additional information retrieval<\/figcaption><\/figure>\nWe have the agent reflect upon whether the answer is useful and grounded in context. If not, it should proceed to\u00a0Transform query<\/code>\u00a0to ask further questions.<\/p>\nThe output\u00a0new_queries<\/code>\u00a0will be a list of new questions that the agent consider useful for obtaining extra information. Given the\u00a0useful_documents<\/code>\u00a0(compensation policies for EU), and\u00a0hypothesis<\/code>\u00a0(need to identify flight distance between Delhi and Munich), it asks questions like\u00a0\u201cWhat is the distance between Delhi and Munich?\u201d<\/strong><\/p>\nNow we are ready to use the\u00a0new_queries<\/code>\u00a0for further retrieval!<\/p>\n\nThe\u00a0transform_query<\/code>\u00a0node will use\u00a0useful_documents<\/code>\u00a0(which are accumulated per iteration, instead of being overwritten) and\u00a0hypothesis<\/code>\u00a0as input for providing the agent directions to ask new questions.<\/p>\nThe new questions will be a list of questions (instead of a single question) separated from the original\u00a0question<\/code>, so that the original question is kept in state, otherwise the agent could lose track of the original question after multiple iterations.<\/p>\n<\/blockquote>\nFinal iteration: Further retrieval with new questions<\/h3>\nIssuing new queries to retrieve extra documents<\/figcaption><\/figure>\nThe output\u00a0new_queries<\/code>\u00a0from\u00a0Transform query<\/code>\u00a0will be passed to the\u00a0Retrieve documents<\/code>\u00a0step, forming a retrieval loop.<\/p>\nSince the question\u00a0\u201cWhat is the distance between Delhi and Munich?\u201d<\/em><\/strong>\u00a0<\/em>is asked, we can expect the flight distance is then retrieved as\u00a0new_documents<\/code>, and subsequently graded as\u00a0useful_documents<\/code>, further used as an input for\u00a0Generate answer<\/code>.<\/p>\n\nThe\u00a0<em>grade_documents<\/em><\/code>\u00a0node will compare the documents against both the original\u00a0<em>question<\/em><\/code>\u00a0and\u00a0<em>new_questions<\/em><\/code>\u00a0list, so that documents that are considered useful for\u00a0<em>new_questions<\/em><\/code>, even if not so for the original\u00a0<em>question<\/em><\/code>, are kept.<\/p>\nThis is because those documents might help answer the original\u00a0<em>question<\/em><\/code>\u00a0indirectly, by being relevant to\u00a0<em>new_questions<\/em><\/code>\u00a0(like \u201c<\/em>What is the distance between Delhi and Munich?\u201d)<\/p>\n<\/blockquote>\nFinal answer!<\/figcaption><\/figure>\nEquipped with this new context about flight distance, the agent is now ready to provide the right answer:\u00a0600 EUR<\/strong>!<\/p>\nNext, let us now dive into the code to see how this multi-agent RAG application is created.<\/p>\n
Implementation<\/h2>\n
The source code can be found\u00a0here<\/a>. Our multi-agent RAG application involves iterations and loops, and LangGraph is a great library for building such complex multi-agent application. If you are not familiar with LangGraph, you are strongly suggested to have a look at\u00a0LangGraph\u2019s Quickstart guide<\/a>\u00a0to understand more about it!<\/p>\nTo keep this article concise, I will focus on the key code snippets only.<\/p>\n
Important note<\/strong>: I am using OpenRouter as the\u00a0Llm<\/a>\u00a0interface, but the code can be easily adapted for other LLM interfaces. Also, while in my code I am using Claude 3.5 Sonnet as model, you can use any LLM as long as it support\u00a0tools<\/code>\u00a0as parameter (check this list\u00a0here<\/a>), so you can also run this with other models, like DeepSeek V3 and OpenAI o1!<\/p>\nState definition<\/h3>\n
In the previous section, I have defined various elements e.g.\u00a0new_documents<\/code>,\u00a0hypothesis<\/code>\u00a0that are to be passed to each step (aka\u00a0Nodes<\/strong>), in LangGraph\u2019s terminology these elements are called\u00a0State<\/strong>.<\/p>\nWe define the State formally with the following snippet.<\/p>\n
from typing import List, Annotated<br>from typing_extensions import TypedDict<br><br>def append_to_list(original: list, new: list) -> list:<br> original.append(new)<br> return original<br><br>def combine_list(original: list, new: list) -> list:<br> return original + new<br><br>class GraphState(TypedDict):<br> \"\"\"<br> Represents the state of our graph.<br><br> Attributes:<br> question: question<br> generation: LLM generation<br> new_documents: newly retrieved documents for the current iteration<br> useful_documents: documents that are considered useful<br> graded_documents: documents that have been graded<br> new_queries: newly generated questions<br> hypothesis: hypothesis<br> \"\"\"<br><br> question: str<br> generation: str<br> new_documents: List[str]<br> useful_documents: Annotated[List[str], combine_list]<br> graded_documents: List[str]<br> new_queries: Annotated[List[str], append_to_list]<br> hypothesis: str<\/code><\/pre>\nGraph definition<\/h3>\n
This is where we combine the different steps to form a \u201cGraph<\/strong>\u201d, which is a representation of our multi-agent application. The definitions of various steps (e.g.\u00a0grade_documents<\/code>) are represented by their respective functions.<\/p>\nfrom langgraph.graph import END, StateGraph, START<br>from langgraph.checkpoint.memory import MemorySaver<br>from IPython.display import Image, display<br><br>workflow = StateGraph(GraphState)<br><br># Define the nodes<br>workflow.add_node(\"retrieve\", retrieve) # retrieve<br>workflow.add_node(\"grade_documents\", grade_documents) # grade documents<br>workflow.add_node(\"generate\", generate) # generatae<br>workflow.add_node(\"transform_query\", transform_query) # transform_query<br><br># Build graph<br>workflow.add_edge(START, \"retrieve\")<br>workflow.add_edge(\"retrieve\", \"grade_documents\")<br>workflow.add_conditional_edges(<br> \"grade_documents\",<br> decide_to_generate,<br> {<br> \"transform_query\": \"transform_query\",<br> \"generate\": \"generate\",<br> },<br>)<br>workflow.add_edge(\"transform_query\", \"retrieve\")<br>workflow.add_conditional_edges(<br> \"generate\",<br> grade_generation_v_documents_and_question,<br> {<br> \"useful\": END,<br> \"not supported\": \"transform_query\",<br> \"not useful\": \"transform_query\",<br> },<br>)<br><br># Compile<br>memory = MemorySaver()<br>app = workflow.compile(checkpointer=memory)<br>display(Image(app.get_graph(xray=True).draw_mermaid_png()))<\/code><\/pre>\nRunning the above code, you should see this graphical representation of our RAG application. Notice how it is essentially equivalent to the graph I have shown in the final iteration of \u201cEnhanced Self-RAG Strategy\u201d!<\/p>\nVisualizing the multi-agent RAG graph<\/figcaption><\/figure>\n\nAfter\u00a0generate<\/code>, if the answer is considered \u201cnot supported\u201d, the agent will proceed to\u00a0transform_query<\/code>\u00a0intead of to\u00a0generate<\/code>\u00a0again, so that the agent will look for additional information rather than trying to regenerate answers based on existing context, which might not suffice for providing a \u201csupported\u201d answer<\/p>\n<\/blockquote>\nNow we are ready to put the multi-agent application to test! With the below code snippet, we ask this question\u00a0how much can I receive if I am denied boarding, for flights from Delhi to Munich?<\/code><\/p>\nfrom pprint import pprint<br>config = {\"configurable\": {\"thread_id\": str(uuid4())}}<br><br># Run<br>inputs = {<br> \"question\": \"how much can I receive if I am denied boarding, for flights from Delhi to Munich?\",<br> }<br>for output in app.stream(inputs, config):<br> for key, value in output.items():<br> # Node<br> pprint(f\"Node '{key}':\")<br> # Optional: print full state at each node<br> # print(app.get_state(config).values)<br> pprint(\"n---n\")<br><br># Final generation<br>pprint(value[\"generation\"])<\/code><\/pre>\nWhile output might vary (sometimes the application provides the answer without any iterations, because it \u201cguessed\u201d the distance between Delhi and Munich), it should look something like this, which shows the application went through multiple rounds of data retrieval for RAG.<\/p>\n
---RETRIEVE---<br>\"Node 'retrieve':\"<br>'n---n'<br>---CHECK DOCUMENT RELEVANCE TO QUESTION---<br>---GRADE: DOCUMENT NOT RELEVANT---<br>---GRADE: DOCUMENT RELEVANT---<br>---GRADE: DOCUMENT NOT RELEVANT---<br>---GRADE: DOCUMENT NOT RELEVANT---<br>---ASSESS GRADED DOCUMENTS---<br>---DECISION: GENERATE---<br>\"Node 'grade_documents':\"<br>'n---n'<br>---GENERATE---<br>---CHECK HALLUCINATIONS---<br>'---DECISION: GENERATION IS NOT GROUNDED IN DOCUMENTS, RE-TRY---'<br>\"Node 'generate':\"<br>'n---n'<br>---TRANSFORM QUERY---<br>\"Node 'transform_query':\"<br>'n---n'<br>---RETRIEVE---<br>\"Node 'retrieve':\"<br>'n---n'<br>---CHECK DOCUMENT RELEVANCE TO QUESTION---<br>---GRADE: DOCUMENT NOT RELEVANT---<br>---GRADE: DOCUMENT NOT RELEVANT---<br>---GRADE: DOCUMENT RELEVANT---<br>---GRADE: DOCUMENT NOT RELEVANT---<br>---GRADE: DOCUMENT NOT RELEVANT---<br>---GRADE: DOCUMENT NOT RELEVANT---<br>---GRADE: DOCUMENT NOT RELEVANT---<br>---ASSESS GRADED DOCUMENTS---<br>---DECISION: GENERATE---<br>\"Node 'grade_documents':\"<br>'n---n'<br>---GENERATE---<br>---CHECK HALLUCINATIONS---<br>---DECISION: GENERATION IS GROUNDED IN DOCUMENTS---<br>---GRADE GENERATION vs QUESTION---<br>---DECISION: GENERATION ADDRESSES QUESTION---<br>\"Node 'generate':\"<br>'n---n'<br>('Based on the context provided, the flight distance from Munich to Delhi is '<br> '5,931 km, which falls into the long-distance category (over 3,500 km). '<br> 'Therefore, if you are denied boarding on a flight from Delhi to Munich '<br> 'operated by an EU airline, you would be eligible for 600 Euro compensation, '<br> 'provided that:n'<br> '1. The flight is operated by an EU airlinen'<br> '2. There is no force majeuren'<br> '3. Other applicable conditions are metn'<br> 'n'<br> \"However, it's important to note that this compensation amount is only valid \"<br> 'if all the required conditions are met as specified in the regulations.')<\/code><\/pre>\nAnd the final answer is what we aimed for!<\/p>\n
Based on the context provided, the flight distance from Munich to Delhi is<br>5,931 km, which falls into the long-distance category (over 3,500 km).<br>Therefore, if you are denied boarding on a flight from Delhi to Munich<br>operated by an EU airline, you would be eligible for 600 Euro compensation,<br>provided that:<br>1. The flight is operated by an EU airline<br>2. There is no force majeure<br>3. Other applicable conditions are met<br><br>However, it's important to note that this compensation amount is only valid<br>if all the required conditions are met as specified in the regulations.<\/code><\/pre>\nInspecting the State, we see how the\u00a0hypothesis<\/code>\u00a0and\u00a0new_queries<\/code>\u00a0enhance the effectiveness of our multi-agent RAG application by mimicking human thinking process.<\/p>\nHypothesis<\/strong><\/p>\nprint(app.get_state(config).values.get('hypothesis',\"\"))<\/code><\/pre>\n--- Output ---<br>To answer this question accurately, I need to determine:<br><br>1. Is this flight operated by an EU airline? (Since Delhi is non-EU and Munich is EU)<br>2. What is the flight distance between Delhi and Munich? (To determine compensation amount)<br>3. Are we dealing with a denied boarding situation due to overbooking? (As opposed to delay\/cancellation)<br><br>From the context, I can find information about compensation amounts based on distance, but I need to verify:<br>- If the flight meets EU compensation eligibility criteria<br>- The exact distance between Delhi and Munich to determine which compensation tier applies (250\u20ac, 400\u20ac, or 600\u20ac)<br>- If denied boarding compensation follows the same amounts as delay compensation<br><br>The context doesn't explicitly state compensation amounts specifically for denied boarding, though it mentions overbooking situations in the EU require offering volunteers re-routing or refund options.<br><br>Would you like me to proceed with the information available, or would you need additional context about denied boarding compensation specifically?<\/code><\/pre>\nNew Queries<\/strong><\/p>\nfor questions_batch in app.get_state(config).values.get('new_queries',\"\"):<br> for q in questions_batch:<br> print(q)<\/code><\/pre>\n--- Output ---<br>What is the flight distance between Delhi and Munich?<br>Does EU denied boarding compensation follow the same amounts as flight delay compensation?<br>Are there specific compensation rules for denied boarding versus flight delays for flights from non-EU to EU destinations?<br>What are the compensation rules when flying with non-EU airlines from Delhi to Munich?<br>What are the specific conditions that qualify as denied boarding under EU regulations?<\/code><\/pre>\nConclusion<\/h2>\n
Simple RAG, while easy to build, might fall short in tackling real-life questions. By incorporating human thinking process into a multi-agent RAG framework, we are making RAG applications much more practical.<\/p>\n
*Unless otherwise noted, all images are by the author<\/em><\/p>\n
\nThe post Supercharge Your RAG with Multi-Agent Self-RAG<\/a> appeared first on Towards Data Science<\/a>.<\/p>\n<\/div>\n
Now, I pass this question to the vector store:\u00a0\u201chow much can I receive if I am denied boarding, for flights from Delhi to Munich?\u201d<\/em><\/strong>.<\/p>\n Unfortunately, they contain only generic flight compensation policies, without telling me how much I can expect when denied boarding from Delhi to Munich specifically. If the RAG agent takes these as the sole context, it can only provide a generic answer about flight compensation policy, without giving the answer we want.<\/p>\n However, while the documents are not immediately useful, there is an important insight contained in the 2nd piece of context:\u00a0compensation varies according to flight distance<\/em><\/strong>. If the RAG agent thinks more like human, it should follow these steps to provide an answer:<\/p>\n This example demonstrates how a simple RAG, in which a single retrieval is made, fall short for several reasons:<\/p>\n Multiple retrievals supplemented with human-like reasoning allow for a more nuanced, comprehensive, and accurate response, adapting to the complexity and depth of user queries.<\/p>\n Here I explain the reasoning process behind this strategy, afterwards I will provide the code to show you how to achieve this!<\/p>\n Note: For readers interested in knowing how my approach differs from the\u00a0original Self-RAG<\/a>, I will describe the discrepancies in quotation boxes like this. But general readers who are unfamiliar with the original Self-RAG can skip them.<\/p>\n<\/blockquote>\n In the below graphs, each circle represents a step (aka\u00a0Node<\/strong>), which is performed by a dedicated agent working on the specific problem. We orchestrate them to form a multi-agent RAG application.<\/p>\n This is just the vanilla RAG approach I described in \u201cReal-life Case\u201d, represented as a graph. After\u00a0 Remember I said in the \u201cReal-life Case\u201d section, that as a next step, the agent should\u00a0\u201creason that compensation varies with flight distance\u201d<\/em><\/strong>? The\u00a0 Given the\u00a0 Notice how the above reasoning resembles human thinking! But still, while these outputs are useful, we need to instruct the agent to use them as input for performing the next document retrieval. Without this, the answer provided in\u00a0 The\u00a0 We have the agent reflect upon whether the answer is useful and grounded in context. If not, it should proceed to\u00a0 The output\u00a0 Now we are ready to use the\u00a0 The\u00a0 The new questions will be a list of questions (instead of a single question) separated from the original\u00a0 The output\u00a0 Since the question\u00a0\u201cWhat is the distance between Delhi and Munich?\u201d<\/em><\/strong>\u00a0<\/em>is asked, we can expect the flight distance is then retrieved as\u00a0 The\u00a0 This is because those documents might help answer the original\u00a0 Equipped with this new context about flight distance, the agent is now ready to provide the right answer:\u00a0600 EUR<\/strong>!<\/p>\n Next, let us now dive into the code to see how this multi-agent RAG application is created.<\/p>\n The source code can be found\u00a0here<\/a>. Our multi-agent RAG application involves iterations and loops, and LangGraph is a great library for building such complex multi-agent application. If you are not familiar with LangGraph, you are strongly suggested to have a look at\u00a0LangGraph\u2019s Quickstart guide<\/a>\u00a0to understand more about it!<\/p>\n To keep this article concise, I will focus on the key code snippets only.<\/p>\n Important note<\/strong>: I am using OpenRouter as the\u00a0Llm<\/a>\u00a0interface, but the code can be easily adapted for other LLM interfaces. Also, while in my code I am using Claude 3.5 Sonnet as model, you can use any LLM as long as it support\u00a0 In the previous section, I have defined various elements e.g.\u00a0 We define the State formally with the following snippet.<\/p>\n This is where we combine the different steps to form a \u201cGraph<\/strong>\u201d, which is a representation of our multi-agent application. The definitions of various steps (e.g.\u00a0 Running the above code, you should see this graphical representation of our RAG application. Notice how it is essentially equivalent to the graph I have shown in the final iteration of \u201cEnhanced Self-RAG Strategy\u201d!<\/p>\n After\u00a0 Now we are ready to put the multi-agent application to test! With the below code snippet, we ask this question\u00a0 While output might vary (sometimes the application provides the answer without any iterations, because it \u201cguessed\u201d the distance between Delhi and Munich), it should look something like this, which shows the application went through multiple rounds of data retrieval for RAG.<\/p>\n And the final answer is what we aimed for!<\/p>\n Inspecting the State, we see how the\u00a0 Hypothesis<\/strong><\/p>\n New Queries<\/strong><\/p>\n Simple RAG, while easy to build, might fall short in tackling real-life questions. By incorporating human thinking process into a multi-agent RAG framework, we are making RAG applications much more practical.<\/p>\n *Unless otherwise noted, all images are by the author<\/em><\/p>\n The post Supercharge Your RAG with Multi-Agent Self-RAG<\/a> appeared first on Towards Data Science<\/a>.<\/p>\n<\/div>\n\u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013<br>Overview of US Flight Compensation Policies To get compensation for delayed flights, you should contact your airline via their customer service or go to the customer service desk. At the same time, you should bear in mind that you will only receive compensation if the delay is not weather-related and is within the carrier`s control. According to the US Department of Transportation, US airlines are not required to compensate you if a flight is cancelled or delayed. You can be compensated if you are bumped or moved from an overbooked flight. If your provider cancels your flight less than two weeks before departure and you decide to cancel your trip entirely, you can receive a refund of both pre-paid baggage fees and your plane ticket. There will be no refund if you choose to continue your journey. In the case of a delayed flight, the airline will rebook you on a different flight. According to federal law, you will not be provided with money or other compensation. Comparative Analysis of EU vs. US Flight Compensation Policies<br>\u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013<br><strong>(AUTHOR-ADDED NOTE: IMPORTANT, PAY ATTENTION TO THIS)<\/strong><br>Short-distance flight delays \u2013 if it is up to 1,500 km, you are due 250 Euro compensation.<br>Medium distance flight delays \u2013 for all the flights between 1,500 and 3,500 km, the compensation should be 400 Euro.<br>Long-distance flight delays \u2013 if it is over 3,500 km, you are due 600 Euro compensation. To receive this kind of compensation, the following conditions must be met; Your flight starts in a non-EU member state or in an EU member state and finishes in an EU member state and is organised by an EU airline. Your flight reaches the final destination with a delay that exceeds three hours. There is no force majeure.<br>\u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013<br>Compensation policies in the EU and US are not the same, which implies that it is worth knowing more about them. While you can always count on Skycop flight cancellation compensation, you should still get acquainted with the information below.<br>\u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013 \u2013<br>Compensation for flight regulations EU: The EU does regulate flight delay compensation, which is known as EU261. US: According to the US Department of Transportation, every airline has its own policies about what should be done for delayed passengers. Compensation for flight delays EU: Just like in the United States, compensation is not provided when the flight is delayed due to uncontrollable reasons. However, there is a clear approach to compensation calculation based on distance. For example, if your flight was up to 1,500 km, you can receive 250 euros. US: There are no federal requirements. That is why every airline sets its own limits for compensation in terms of length. However, it is usually set at three hours. Overbooking EU: In the EU, they call for volunteers if the flight is overbooked. These people are entitled to a choice of: Re-routing to their final destination at the earliest opportunity. Refund of their ticket cost within a week if not travelling. Re-routing at a later date at the person`s convenience.<\/code><\/pre>\n\n
\n
Multi-Agent Self-RAG<\/h2>\n
\n
1st iteration: Simple RAG<\/h3>\n
Retrieve documents<\/code>, the\u00a0new_documents<\/code>\u00a0will be used as input for\u00a0Generate Answer<\/code>. Nothing special, but it serves as our starting point.<\/p>\n2nd iteration: Digest documents with \u201cGrade documents\u201d<\/h3>\n
Grade documents<\/code>\u00a0step is exactly for this purpose.<\/p>\nnew_documents<\/code>, the agent will try to output two items:<\/p>\n\n
useful_documents<\/code>: Comparing the question asked, it determines if the documents are useful, and retain a memory for those deemed useful for future reference. As an example, since our question does not concern compensation policies for US, documents describing those are discarded, leaving only those for EU<\/li>\nhypothesis<\/code>: Based on the documents, the agent forms a hypothesis about how the question can be answered, that is, flight distance needs to be identified<\/li>\n<\/ol>\nGenerate answer<\/code>\u00a0is still not useful.<\/p>\n\n
useful_documents<\/code>\u00a0are appended for each document retrieval loop, instead of being overwritten, to keep a memory of documents that are previously deemed useful.\u00a0hypothesis<\/code>\u00a0is formed from\u00a0useful_documents<\/code>\u00a0and\u00a0new_documents<\/code>\u00a0to provide an \u201cabstract reasoning\u201d to inform how query is to be transformed subsequently.<\/p>\nhypothesis<\/code>\u00a0is especially useful when no useful documents can be identified initially, as the agent can still form hypothesis from documents not immediately deemed as useful \/ only bearing indirect relationship to the question at hand, for informing what questions to ask next<\/p>\n<\/blockquote>\n3rd iteration: Brainstorm new questions to ask<\/h3>\n
Transform query<\/code>\u00a0to ask further questions.<\/p>\nnew_queries<\/code>\u00a0will be a list of new questions that the agent consider useful for obtaining extra information. Given the\u00a0useful_documents<\/code>\u00a0(compensation policies for EU), and\u00a0hypothesis<\/code>\u00a0(need to identify flight distance between Delhi and Munich), it asks questions like\u00a0\u201cWhat is the distance between Delhi and Munich?\u201d<\/strong><\/p>\nnew_queries<\/code>\u00a0for further retrieval!<\/p>\n\n
transform_query<\/code>\u00a0node will use\u00a0useful_documents<\/code>\u00a0(which are accumulated per iteration, instead of being overwritten) and\u00a0hypothesis<\/code>\u00a0as input for providing the agent directions to ask new questions.<\/p>\nquestion<\/code>, so that the original question is kept in state, otherwise the agent could lose track of the original question after multiple iterations.<\/p>\n<\/blockquote>\nFinal iteration: Further retrieval with new questions<\/h3>\n
new_queries<\/code>\u00a0from\u00a0Transform query<\/code>\u00a0will be passed to the\u00a0Retrieve documents<\/code>\u00a0step, forming a retrieval loop.<\/p>\nnew_documents<\/code>, and subsequently graded as\u00a0useful_documents<\/code>, further used as an input for\u00a0Generate answer<\/code>.<\/p>\n\n
<em>grade_documents<\/em><\/code>\u00a0node will compare the documents against both the original\u00a0<em>question<\/em><\/code>\u00a0and\u00a0<em>new_questions<\/em><\/code>\u00a0list, so that documents that are considered useful for\u00a0<em>new_questions<\/em><\/code>, even if not so for the original\u00a0<em>question<\/em><\/code>, are kept.<\/p>\n<em>question<\/em><\/code>\u00a0indirectly, by being relevant to\u00a0<em>new_questions<\/em><\/code>\u00a0(like \u201c<\/em>What is the distance between Delhi and Munich?\u201d)<\/p>\n<\/blockquote>\nImplementation<\/h2>\n
tools<\/code>\u00a0as parameter (check this list\u00a0here<\/a>), so you can also run this with other models, like DeepSeek V3 and OpenAI o1!<\/p>\nState definition<\/h3>\n
new_documents<\/code>,\u00a0hypothesis<\/code>\u00a0that are to be passed to each step (aka\u00a0Nodes<\/strong>), in LangGraph\u2019s terminology these elements are called\u00a0State<\/strong>.<\/p>\nfrom typing import List, Annotated<br>from typing_extensions import TypedDict<br><br>def append_to_list(original: list, new: list) -> list:<br> original.append(new)<br> return original<br><br>def combine_list(original: list, new: list) -> list:<br> return original + new<br><br>class GraphState(TypedDict):<br> \"\"\"<br> Represents the state of our graph.<br><br> Attributes:<br> question: question<br> generation: LLM generation<br> new_documents: newly retrieved documents for the current iteration<br> useful_documents: documents that are considered useful<br> graded_documents: documents that have been graded<br> new_queries: newly generated questions<br> hypothesis: hypothesis<br> \"\"\"<br><br> question: str<br> generation: str<br> new_documents: List[str]<br> useful_documents: Annotated[List[str], combine_list]<br> graded_documents: List[str]<br> new_queries: Annotated[List[str], append_to_list]<br> hypothesis: str<\/code><\/pre>\nGraph definition<\/h3>\n
grade_documents<\/code>) are represented by their respective functions.<\/p>\nfrom langgraph.graph import END, StateGraph, START<br>from langgraph.checkpoint.memory import MemorySaver<br>from IPython.display import Image, display<br><br>workflow = StateGraph(GraphState)<br><br># Define the nodes<br>workflow.add_node(\"retrieve\", retrieve) # retrieve<br>workflow.add_node(\"grade_documents\", grade_documents) # grade documents<br>workflow.add_node(\"generate\", generate) # generatae<br>workflow.add_node(\"transform_query\", transform_query) # transform_query<br><br># Build graph<br>workflow.add_edge(START, \"retrieve\")<br>workflow.add_edge(\"retrieve\", \"grade_documents\")<br>workflow.add_conditional_edges(<br> \"grade_documents\",<br> decide_to_generate,<br> {<br> \"transform_query\": \"transform_query\",<br> \"generate\": \"generate\",<br> },<br>)<br>workflow.add_edge(\"transform_query\", \"retrieve\")<br>workflow.add_conditional_edges(<br> \"generate\",<br> grade_generation_v_documents_and_question,<br> {<br> \"useful\": END,<br> \"not supported\": \"transform_query\",<br> \"not useful\": \"transform_query\",<br> },<br>)<br><br># Compile<br>memory = MemorySaver()<br>app = workflow.compile(checkpointer=memory)<br>display(Image(app.get_graph(xray=True).draw_mermaid_png()))<\/code><\/pre>\n\n
generate<\/code>, if the answer is considered \u201cnot supported\u201d, the agent will proceed to\u00a0transform_query<\/code>\u00a0intead of to\u00a0generate<\/code>\u00a0again, so that the agent will look for additional information rather than trying to regenerate answers based on existing context, which might not suffice for providing a \u201csupported\u201d answer<\/p>\n<\/blockquote>\nhow much can I receive if I am denied boarding, for flights from Delhi to Munich?<\/code><\/p>\nfrom pprint import pprint<br>config = {\"configurable\": {\"thread_id\": str(uuid4())}}<br><br># Run<br>inputs = {<br> \"question\": \"how much can I receive if I am denied boarding, for flights from Delhi to Munich?\",<br> }<br>for output in app.stream(inputs, config):<br> for key, value in output.items():<br> # Node<br> pprint(f\"Node '{key}':\")<br> # Optional: print full state at each node<br> # print(app.get_state(config).values)<br> pprint(\"n---n\")<br><br># Final generation<br>pprint(value[\"generation\"])<\/code><\/pre>\n---RETRIEVE---<br>\"Node 'retrieve':\"<br>'n---n'<br>---CHECK DOCUMENT RELEVANCE TO QUESTION---<br>---GRADE: DOCUMENT NOT RELEVANT---<br>---GRADE: DOCUMENT RELEVANT---<br>---GRADE: DOCUMENT NOT RELEVANT---<br>---GRADE: DOCUMENT NOT RELEVANT---<br>---ASSESS GRADED DOCUMENTS---<br>---DECISION: GENERATE---<br>\"Node 'grade_documents':\"<br>'n---n'<br>---GENERATE---<br>---CHECK HALLUCINATIONS---<br>'---DECISION: GENERATION IS NOT GROUNDED IN DOCUMENTS, RE-TRY---'<br>\"Node 'generate':\"<br>'n---n'<br>---TRANSFORM QUERY---<br>\"Node 'transform_query':\"<br>'n---n'<br>---RETRIEVE---<br>\"Node 'retrieve':\"<br>'n---n'<br>---CHECK DOCUMENT RELEVANCE TO QUESTION---<br>---GRADE: DOCUMENT NOT RELEVANT---<br>---GRADE: DOCUMENT NOT RELEVANT---<br>---GRADE: DOCUMENT RELEVANT---<br>---GRADE: DOCUMENT NOT RELEVANT---<br>---GRADE: DOCUMENT NOT RELEVANT---<br>---GRADE: DOCUMENT NOT RELEVANT---<br>---GRADE: DOCUMENT NOT RELEVANT---<br>---ASSESS GRADED DOCUMENTS---<br>---DECISION: GENERATE---<br>\"Node 'grade_documents':\"<br>'n---n'<br>---GENERATE---<br>---CHECK HALLUCINATIONS---<br>---DECISION: GENERATION IS GROUNDED IN DOCUMENTS---<br>---GRADE GENERATION vs QUESTION---<br>---DECISION: GENERATION ADDRESSES QUESTION---<br>\"Node 'generate':\"<br>'n---n'<br>('Based on the context provided, the flight distance from Munich to Delhi is '<br> '5,931 km, which falls into the long-distance category (over 3,500 km). '<br> 'Therefore, if you are denied boarding on a flight from Delhi to Munich '<br> 'operated by an EU airline, you would be eligible for 600 Euro compensation, '<br> 'provided that:n'<br> '1. The flight is operated by an EU airlinen'<br> '2. There is no force majeuren'<br> '3. Other applicable conditions are metn'<br> 'n'<br> \"However, it's important to note that this compensation amount is only valid \"<br> 'if all the required conditions are met as specified in the regulations.')<\/code><\/pre>\nBased on the context provided, the flight distance from Munich to Delhi is<br>5,931 km, which falls into the long-distance category (over 3,500 km).<br>Therefore, if you are denied boarding on a flight from Delhi to Munich<br>operated by an EU airline, you would be eligible for 600 Euro compensation,<br>provided that:<br>1. The flight is operated by an EU airline<br>2. There is no force majeure<br>3. Other applicable conditions are met<br><br>However, it's important to note that this compensation amount is only valid<br>if all the required conditions are met as specified in the regulations.<\/code><\/pre>\nhypothesis<\/code>\u00a0and\u00a0new_queries<\/code>\u00a0enhance the effectiveness of our multi-agent RAG application by mimicking human thinking process.<\/p>\nprint(app.get_state(config).values.get('hypothesis',\"\"))<\/code><\/pre>\n--- Output ---<br>To answer this question accurately, I need to determine:<br><br>1. Is this flight operated by an EU airline? (Since Delhi is non-EU and Munich is EU)<br>2. What is the flight distance between Delhi and Munich? (To determine compensation amount)<br>3. Are we dealing with a denied boarding situation due to overbooking? (As opposed to delay\/cancellation)<br><br>From the context, I can find information about compensation amounts based on distance, but I need to verify:<br>- If the flight meets EU compensation eligibility criteria<br>- The exact distance between Delhi and Munich to determine which compensation tier applies (250\u20ac, 400\u20ac, or 600\u20ac)<br>- If denied boarding compensation follows the same amounts as delay compensation<br><br>The context doesn't explicitly state compensation amounts specifically for denied boarding, though it mentions overbooking situations in the EU require offering volunteers re-routing or refund options.<br><br>Would you like me to proceed with the information available, or would you need additional context about denied boarding compensation specifically?<\/code><\/pre>\nfor questions_batch in app.get_state(config).values.get('new_queries',\"\"):<br> for q in questions_batch:<br> print(q)<\/code><\/pre>\n--- Output ---<br>What is the flight distance between Delhi and Munich?<br>Does EU denied boarding compensation follow the same amounts as flight delay compensation?<br>Are there specific compensation rules for denied boarding versus flight delays for flights from non-EU to EU destinations?<br>What are the compensation rules when flying with non-EU airlines from Delhi to Munich?<br>What are the specific conditions that qualify as denied boarding under EU regulations?<\/code><\/pre>\nConclusion<\/h2>\n
\n