{"id":2186,"date":"2025-03-04T07:02:30","date_gmt":"2025-03-04T07:02:30","guid":{"rendered":"https:\/\/mailitics.com\/index.php\/2025\/03\/04\/llm-rag-creating-an-ai-powered-file-reader-assistant\/"},"modified":"2025-03-04T07:02:30","modified_gmt":"2025-03-04T07:02:30","slug":"llm-rag-creating-an-ai-powered-file-reader-assistant","status":"publish","type":"post","link":"https:\/\/mailitics.com\/index.php\/2025\/03\/04\/llm-rag-creating-an-ai-powered-file-reader-assistant\/","title":{"rendered":"LLM + RAG: Creating an AI-Powered File Reader Assistant"},"content":{"rendered":"

LLM + RAG: Creating an AI-Powered File Reader Assistant
\n \t

\n
<\/BR>
\n
\n \t

\n
<\/BR><\/p>\n

\n

Introduction<\/h2>\n

AI is everywhere.\u00a0<\/p>\n

It is hard not to interact at least once a day with a Large Language Model (LLM). The chatbots are here to stay. They\u2019re in your apps, they help you write better, they compose emails, they read emails\u2026well, they do a lot.<\/p>\n

And I don\u2019t think that that is bad. In fact, my opinion is the other way \u2013 at least so far. I defend and advocate for the use of AI in our daily lives because, let\u2019s agree, it makes everything much easier.<\/p>\n

I don\u2019t have to spend time double-reading a document to find punctuation problems or type. AI does that for me. I don\u2019t waste time writing that follow-up email every single Monday. AI does that for me. I don\u2019t need to read a huge and boring contract when I have an AI to summarize the main takeaways and action points to me!<\/p>\n

These are only some of AI\u2019s great uses. If you\u2019d like to know more use cases of LLMs to make our lives easier, I wrote a whole book about them.<\/p>\n

Now, thinking as a data scientist and looking at the technical side, not everything is that bright and shiny.\u00a0<\/p>\n

LLMs are great for several general use cases that apply to anyone or any company. For example, coding, summarizing, or answering questions about general content created until the training cutoff date. However, when it comes to specific business applications, for a single purpose, or something new that didn\u2019t make the cutoff date, that is when the models won\u2019t be that useful if used out-of-the-box<\/em> \u2013 meaning, they will not know the answer. Thus, it will need adjustments.<\/p>\n

Training an LLM model can take months and millions of dollars. What is even worse is that if we don\u2019t adjust and tune the model to our purpose, there will be unsatisfactory results or hallucinations (when the model\u2019s response doesn\u2019t make sense given our query).<\/p>\n

So what is the solution, then? Spending a lot of money retraining the model to include our data?<\/p>\n

Not really. That\u2019s when the Retrieval-Augmented Generation (RAG) becomes useful.<\/p>\n

RAG is a framework that combines getting information from an external knowledge base with large language models (LLMs). It helps AI models produce more accurate and relevant responses.<\/em><\/p>\n

Let\u2019s learn more about RAG next.<\/p>\n

What is RAG?<\/h2>\n

Let me tell you a story to illustrate the concept.<\/p>\n

I love movies. For some time in the past, I knew which movies were competing for the best movie category at the Oscars or the best actors and actresses. And I would certainly know which ones got the statue for that year. But now I am all rusty on that subject. If you asked me who was competing, I would not know. And even if I tried to answer you, I would give you a weak response.\u00a0<\/p>\n

So, to provide you with a quality response, I will do what everybody else does: search for the information online, obtain it, and then give it to you. What I just did is the same idea as the RAG: I obtained data from an external database to give you an answer.<\/p>\n

When we enhance the LLM with a content store<\/em> where it can go and retrieve<\/strong> data to augment<\/strong> (increase) its knowledge base, that is the RAG framework in action.<\/p>\n

RAG is like creating a content store where the model can enhance its knowledge and respond more accurately.<\/em><\/p>\n

\"Diagram:
User prompt about Content C. LLM retrieves external content to aggregate to the answer. Image by the author.<\/figcaption><\/figure>\n

Summarizing:<\/p>\n

    \n
  1. Uses search algorithms to query external data sources, such as databases, knowledge bases, and web pages.<\/li>\n
  2. Pre-processes the retrieved information.<\/li>\n
  3. Incorporates the pre-processed information into the LLM.<\/li>\n<\/ol>\n

    Why use RAG?<\/h2>\n

    Now that we know what the RAG framework is let\u2019s understand why we should be using it.<\/p>\n

    Here are some of the benefits:<\/p>\n