{"id":3628,"date":"2025-05-07T07:02:21","date_gmt":"2025-05-07T07:02:21","guid":{"rendered":"https:\/\/mailitics.com\/index.php\/2025\/05\/07\/retrieval-augmented-classification-improving-text-classification-with-external-knowledge\/"},"modified":"2025-05-07T07:02:21","modified_gmt":"2025-05-07T07:02:21","slug":"retrieval-augmented-classification-improving-text-classification-with-external-knowledge","status":"publish","type":"post","link":"https:\/\/mailitics.com\/index.php\/2025\/05\/07\/retrieval-augmented-classification-improving-text-classification-with-external-knowledge\/","title":{"rendered":"Retrieval Augmented Classification: Improving Text Classification with External Knowledge"},"content":{"rendered":"

Retrieval Augmented Classification: Improving Text Classification with External Knowledge
\n \t

\n
<\/BR>
\n
\n \t

\n
<\/BR><\/p>\n

\n

Text<\/mdspan> Classification<\/a> stands as one of the most basic yet most important applications of natural language processing. It has a vital role in many real-world applications that go from filtering unwanted emails like spam, detecting product categories or classifying user intent in a chat-bot application. The default way of building text classifiers is to gather large amounts of labeled data, meaning input texts and their corresponding labels, and then training a custom Machine Learning model. Things changed a bit as LLMs became more powerful, where you can often get decent performance by using general purpose large language models as zero-shot or few-shot classifiers, significantly reducing the time-to-deployment of text classification services. However, the accuracy can lag behind custom built models and is highly dependent on crafting custom prompts to better define the classification task to the LLM. In this blog, we aim at minimizing the gap between custom ML models for classification and general purpose LLMs while also minimizing the effort needed in adapting the LLM prompt to your task.<\/p>\n

LLMs vs Custom ML models for text classification<\/h2>\n

Pros:<\/strong><\/p>\n

Let\u2019s first explore the pro and cons of each of the two approaches to do text classification.<\/p>\n

Large language models as general purpose classifiers:<\/strong><\/p>\n

    \n
  1. High generalization ability given the vast pre-training corpus and reasoning abilities of the LLM.<\/li>\n
  2. A single general purpose LLM can handle multiple classifications tasks without the need to deploy a model for each.<\/li>\n
  3. As Llms<\/a> continue to improve, you can potentially enhance accuracy with minimal effort simply by adopting newer, more powerful models as they become available.<\/li>\n
  4. The availability of most LLMs as managed services significantly reduces the deployment knowledge and effort required to get started.<\/li>\n
  5. LLMs often outperform custom ML models in low-data scenarios where labeled data is limited or costly to obtain.<\/li>\n
  6. LLMs generalize to multiple languages.<\/li>\n
  7. LLMs can be cheaper when having low or unpredictable volumes of predictions if you pay per token.<\/li>\n
  8. Class definitions can be changed dynamically without retraining by simply modifying the prompts.<\/li>\n<\/ol>\n

    Cons:<\/strong><\/p>\n

      \n
    1. LLMs are prone to hallucinations.<\/li>\n
    2. LLMs can be slow, or at least slower than small custom ML models.<\/li>\n
    3. They require prompt engineering effort.<\/li>\n
    4. High-throughput applications using LLMs-as-a-service may quickly encounter quota limitations.<\/li>\n
    5. This approach becomes less effective with a very large number of potential classes due to context size constraints. Defining all the classes would consume a significant portion of the available and effective input context.<\/li>\n
    6. LLMs usually have worse accuracy than custom models in the high data regime.<\/li>\n<\/ol>\n

      Custom Machine Learning<\/a> models:<\/strong><\/p>\n

      Pros:<\/strong><\/p>\n

        \n
      1. Efficient and fast.<\/li>\n
      2. More flexible in architecture choice, training and serving method.<\/li>\n
      3. Ability to add interpretability and uncertainty estimation aspects to the model.<\/li>\n
      4. Higher accuracy in the high data regime.<\/li>\n
      5. You keep control of your model and serving infrastructure.<\/li>\n<\/ol>\n

        Cons:<\/strong><\/p>\n

          \n
        1. Requires frequent re-trainings to adapt to new data or distribution changes.<\/li>\n
        2. May need significant amounts of labeled data.<\/li>\n
        3. Limited generalization.<\/li>\n
        4. Sensitive to out-of-domain vocabulary or formulations.<\/li>\n
        5. Requires MLOps knowledge for deployment.<\/li>\n<\/ol>\n

          Bridging the gap between custom text classifier and LLMs:<\/h2>\n

          Let\u2019s work on a way to keep the pros of using LLMs for classification while alleviating some of the cons. We will take inspiration from RAG and use a prompting technique called few-shot prompting.<\/p>\n

          Let\u2019s define both:<\/p>\n

          RAG<\/strong><\/p>\n

          Retrieval Augmented Generation is a popular method that augments the LLM context with external knowledge before asking a question. This reduces the likelihood of hallucination and improves the quality of the responses.<\/p>\n

          Few-shot prompting<\/strong><\/p>\n

          In each classification task, we show the LLM examples of inputs and expected outputs as part of the prompt to help it understand the task.<\/p>\n

          Now, the main idea of this project is mixing both. We dynamically fetch examples that are the most similar to the text query to be classified and inject them as few-shot example prompts. We also limit the scope of possible classes dynamically using those of the K-nearest neighbors. This frees up a significant amount of tokens in the input context when working with a classification problem with a large number of possible classes.<\/p>\n

          Here is how that would work:<\/p>\n

          <\/a><\/a><\/p>\n

          \"\"<\/figure>\n

          Let\u2019s go through the practical steps of getting this approach to run:<\/p>\n