{"id":3899,"date":"2025-05-17T07:04:47","date_gmt":"2025-05-17T07:04:47","guid":{"rendered":"https:\/\/mailitics.com\/index.php\/2025\/05\/17\/agentic-ai-102-guardrails-and-agent-evaluation\/"},"modified":"2025-05-17T07:04:47","modified_gmt":"2025-05-17T07:04:47","slug":"agentic-ai-102-guardrails-and-agent-evaluation","status":"publish","type":"post","link":"https:\/\/mailitics.com\/index.php\/2025\/05\/17\/agentic-ai-102-guardrails-and-agent-evaluation\/","title":{"rendered":"Agentic AI 102: Guardrails and Agent Evaluation"},"content":{"rendered":"

Agentic AI 102: Guardrails and Agent Evaluation
\n \t

\n
<\/BR>
\n
\n \t

\n
<\/BR><\/p>\n

\n

Introduction<\/mdspan><\/h2>\n

In the first post of this series (Agentic AI 101: Starting Your Journey Building AI Agents<\/a>), we talked about the fundamentals of creating AI Agents and introduced concepts like reasoning, memory, and tools.<\/p>\n

Of course, that first post touched only the surface of this new area of the data industry. There is so much more that can be done, and we are going to learn more along the way in this series.<\/p>\n

So, it is time to take one step further.<\/p>\n

In this post, we will cover three topics:<\/p>\n

    \n
  1. \nGuardrails<\/strong>: these are safe blocks that prevent a Large Language Model (LLM) from responding about some topics.<\/li>\n
  2. \nAgent Evaluation<\/strong>: Have you ever thought about how accurate the responses from LLM are? I bet you did. So we will see the main ways to measure that.<\/li>\n
  3. \nMonitoring<\/strong>: We will also learn about the built-in monitoring app in Agno\u2019s framework.<\/li>\n<\/ol>\n

    We shall begin now.<\/p>\n

    Guardrails<\/h2>\n

    Our first topic is the simplest, in my opinion. Guardrails are rules that will keep an AI agent from responding to a given topic or list of topics.<\/p>\n

    I believe there is a good chance that you have ever asked something to ChatGPT or Gemini and received a response like \u201cI can\u2019t talk about this topic\u201d, or \u201cPlease consult a professional specialist\u201d, something like that. Usually, that occurs with sensitive topics like health advice, psychological conditions, or financial advice.<\/p>\n

    Those blocks are safeguards to prevent people from hurting themselves, harming their health, or their pockets. As we know, LLMs are trained on massive amounts of text, ergo inheriting a lot of bad content with it, which could easily lead to bad advice in those areas for people. And I didn\u2019t even mention hallucinations! <\/p>\n

    Think about how many stories there are of people who lost money by following investment tips from online forums. Or how many people took the wrong medicine because they read about it on the internet<\/em>.<\/p>\n

    Well, I guess you got the point. We must prevent our agents from talking about certain topics or taking certain actions. For that, we will use guardrails.<\/p>\n

    The best framework I found to impose those blocks is Guardrails AI [1]<\/a>. There, you will see a hub full of predefined rules that a response must follow in order to pass and be displayed to the user.<\/p>\n

    To get started quickly, first go to this link [2] and get an API key. Then, install the package. Next, type the guardrails setup command. It will ask you a couple of questions that you can respond n (for No), and it will ask you to enter the API Key generated.<\/p>\n

    pip install guardrails-ai\nguardrails configure<\/code><\/pre>\n
    Once that is completed, go to the Guardrails AI Hub [3]<\/a> and choose one that you need. Every guardrail has instructions on how to implement it. Basically, you install it via the command line and then use it like a module in Python.<\/p>\n
    For this example, we\u2019re choosing one called Restrict to Topic<\/em> [4], which, as its name says, lets the user talk only about what\u2019s in the list. So, go back to the terminal and install it using the code below.<\/p>\n
    guardrails hub install hub:\/\/tryolabs\/restricttotopic<\/code><\/pre>\n
    Next, let\u2019s open our Python script and import some modules.<\/p>\n
    # Imports\nfrom agno.agent import Agent\nfrom agno.models.google import Gemini\nimport os\n\n# Import Guard and Validator\nfrom guardrails import Guard\nfrom guardrails.hub import RestrictToTopic\n<\/code><\/pre>\n
    Next, we create the guard. We will restrict our agent to talk only about sports<\/em> or the weather<\/em>. And we are restricting it to talk about stocks<\/em>.<\/p>\n
    # Setup Guard\nguard = Guard().use(\n    RestrictToTopic(\n        valid_topics=[\"sports\", \"weather\"],\n        invalid_topics=[\"stocks\"],\n        disable_classifier=True,\n        disable_llm=False,\n        on_fail=\"filter\"\n    )\n)<\/code><\/pre>\n
    Now we can run the agent and the guard.<\/p>\n
    # Create agent\nagent = Agent(\n    model= Gemini(id=\"gemini-1.5-flash\",\n                  api_key = os.environ.get(\"GEMINI_API_KEY\")),\n    description= \"An assistant agent\",\n    instructions= [\"Be sucint. Reply in maximum two sentences\"],\n    markdown= True\n    )\n\n# Run the agent\nresponse = agent.run(\"What's the ticker symbol for Apple?\").content\n\n# Run agent with validation\nvalidation_step = guard.validate(response)\n\n# Print validated response\nif validation_step.validation_passed:\n    print(response)\nelse:\n    print(\"Validation Failed\", validation_step.validation_summaries[0].failure_reason)<\/code><\/pre>\n
    This is the response when we ask about a stock symbol.<\/p>\n
    Validation Failed Invalid topics found: ['stocks']<\/code><\/pre>\n
    If I ask about a topic that is not on the valid_topics<\/code> list, I will also see a block.<\/p>\n
    \"What's the number one soda drink?\"\nValidation Failed No valid topic was found.<\/code><\/pre>\n
    Finally, let\u2019s ask about sports.<\/p>\n
    \"Who is Michael Jordan?\"\nMichael Jordan is a former professional basketball player widely considered one of \nthe greatest of all time.  He won six NBA championships with the Chicago Bulls.<\/code><\/pre>\n
    And we saw a response this time, as it is a valid topic.<\/p>\n
    Let\u2019s move on to the evaluation of agents now.<\/p>\n
    Agent Evaluation<\/h2>\n
    Since I started studying LLMs and Agentic Ai<\/a>, one of my main questions has been about model evaluation. Unlike traditional Data Science Modeling, where you have structured metrics that are adequate for each case, for AI Agents, this is more blurry.<\/p>\n
    Fortunately, the developer community is pretty quick in finding solutions for almost everything, and so they created this nice package for LLMs evaluation: deepeval<\/code>.<\/p>\n
    DeepEval [5]<\/a> is a library created by Confident AI that gathers many methods to evaluate LLMs and AI Agents. In this section, let\u2019s learn a couple of the main methods, just so we can build some intuition on the subject, and also because the library is quite extensive.<\/p>\n
    The first evaluation is the most basic we can use, and it is called G-Eval<\/code>. As AI tools like ChatGPT become more common in everyday tasks, we have to make sure they\u2019re giving helpful and accurate responses. That\u2019s where G-Eval from the DeepEval Python package comes in. <\/p>\n
    G-Eval<\/strong> is like a smart reviewer that uses another AI model to evaluate how well a chatbot or AI assistant is performing. For example. My agent runs Gemini, and I am using OpenAI to assess it. This method takes a more advanced approach than a human one by asking an AI to \u201cgrade\u201d another AI\u2019s answers based on things like relevance<\/em>, correctness<\/em>, and clarity<\/em>. <\/p>\n
    It\u2019s a nice way to test and improve generative AI systems in a more scalable way. Let\u2019s quickly code an example. We will import the modules, create a prompt, a simple chat agent, and ask it about a description of the weather for the month of May in NYC.<\/p>\n
    # Imports\nfrom agno.agent import Agent\nfrom agno.models.google import Gemini\nimport os\n# Evaluation Modules\nfrom deepeval.test_case import LLMTestCase, LLMTestCaseParams\nfrom deepeval.metrics import GEval\n\n# Prompt\nprompt = \"Describe the weather in NYC for May\"\n\n# Create agent\nagent = Agent(\n    model= Gemini(id=\"gemini-1.5-flash\",\n                  api_key = os.environ.get(\"GEMINI_API_KEY\")),\n    description= \"An assistant agent\",\n    instructions= [\"Be sucint\"],\n    markdown= True,\n    monitoring= True\n    )\n\n# Run agent\nresponse = agent.run(prompt)\n\n# Print response\nprint(response.content)<\/code><\/pre>\n
    It responds: \u201cMild, with average highs in the 60s\u00b0F and lows in the 50s\u00b0F. Expect some rain<\/em>\u201c. <\/p>\n
    Nice. Seems pretty good to me.<\/p>\n
    But how can we put a number on it and show a potential manager or client how our agent is doing?<\/p>\n
    Here is how:<\/p>\n
      \n
    1. Create a test case passing the prompt<\/code> and the response<\/code> to the LLMTestCase<\/code> class.<\/li>\n
    2. Create a metric. We will use the method GEval<\/code> and add a prompt for the model to test it for coherence<\/strong>, and then I give it the meaning of what coherence is to me.<\/li>\n
    3. Give the output as evaluation_params<\/code>.<\/li>\n
    4. Run the measure<\/code> method and get the score<\/code> and reason<\/code> from it.<\/li>\n<\/ol>\n
      # Test Case\ntest_case = LLMTestCase(input=prompt, actual_output=response)\n\n# Setup the Metric\ncoherence_metric = GEval(\n    name=\"Coherence\",\n    criteria=\"Coherence. The agent can answer the prompt and the response makes sense.\",\n    evaluation_params=[LLMTestCaseParams.ACTUAL_OUTPUT]\n)\n\n# Run the metric\ncoherence_metric.measure(test_case)\nprint(coherence_metric.score)\nprint(coherence_metric.reason)<\/code><\/pre>\n
      The output looks like this.<\/p>\n
      0.9\nThe response directly addresses the prompt about NYC weather in May, \nmaintains logical consistency, flows naturally, and uses clear language. \nHowever, it could be slightly more detailed.<\/code><\/pre>\n
      0.9 seems pretty good, given that the default threshold is 0.5.<\/p>\n
      If you want to check the logs, use this next snippet.<\/p>\n
      # Check the logs\nprint(coherence_metric.verbose_logs)<\/code><\/pre>\n
      Here\u2019s the response.<\/p>\n
      Criteria:\nCoherence. The agent can answer the prompt and the response makes sense.\n\nEvaluation Steps:\n[\n    \"Assess whether the response directly addresses the prompt; if it aligns,\n it scores higher on coherence.\",\n    \"Evaluate the logical flow of the response; responses that present ideas\n in a clear, organized manner rank better in coherence.\",\n    \"Consider the relevance of examples or evidence provided; responses that \ninclude pertinent information enhance their coherence.\",\n    \"Check for clarity and consistency in terminology; responses that maintain\n clear language without contradictions achieve a higher coherence rating.\"\n]<\/code><\/pre>\n
      Very nice. Now let us learn about another interesting use case, which is the evaluation of task completion for AI Agents<\/strong>. Elaborating a little more, how our agent is doing when it is requested to perform a task, and how much of it the agent can deliver.<\/p>\n
      First, we are creating a simple agent that can access Wikipedia and summarize the topic of the query.<\/p>\n
      # Imports\nfrom agno.agent import Agent\nfrom agno.models.google import Gemini\nfrom agno.tools.wikipedia import WikipediaTools\nimport os\nfrom deepeval.test_case import LLMTestCase, ToolCall\nfrom deepeval.metrics import TaskCompletionMetric\nfrom deepeval import evaluate\n\n# Prompt\nprompt = \"Search wikipedia for 'Time series analysis' and summarize the 3 main points\"\n\n# Create agent\nagent = Agent(\n    model= Gemini(id=\"gemini-2.0-flash\",\n                  api_key = os.environ.get(\"GEMINI_API_KEY\")),\n    description= \"You are a researcher specialized in searching the wikipedia.\",\n    tools= [WikipediaTools()],\n    show_tool_calls= True,\n    markdown= True,\n    read_tool_call_history= True\n    )\n\n# Run agent\nresponse = agent.run(prompt)\n\n# Print response\nprint(response.content)<\/code><\/pre>\n
      The result looks very good. Let\u2019s evaluate it using the TaskCompletionMetric<\/code> class.<\/p>\n
      # Create a Metric\nmetric = TaskCompletionMetric(\n    threshold=0.7,\n    model=\"gpt-4o-mini\",\n    include_reason=True\n)\n\n# Test Case\ntest_case = LLMTestCase(\n    input=prompt,\n    actual_output=response.content,\n    tools_called=[ToolCall(name=\"wikipedia\")]\n    )\n\n# Evaluate\nevaluate(test_cases=[test_case], metrics=[metric])<\/code><\/pre>\n
      Output, including the agent\u2019s response.<\/p>\n
      ======================================================================\n\nMetrics Summary\n\n  - \"\u2705\" Task Completion (score: 1.0, threshold: 0.7, strict: False, \nevaluation model: gpt-4o-mini, \nreason: The system successfully searched for 'Time series analysis' \non Wikipedia and provided a clear summary of the 3 main points, \nfully aligning with the user's goal., error: None)\n\nFor test case:\n\n  - input: Search wikipedia for 'Time series analysis' and summarize the 3 main points\n  - actual output: Here are the 3 main points about Time series analysis based on the\n Wikipedia search:\n\n1.  **Definition:** A time series is a sequence of data points indexed in time order,\n often taken at successive, equally spaced points in time.\n2.  **Applications:** Time series analysis is used in various fields like statistics,\n signal processing, econometrics, weather forecasting, and more, wherever temporal \nmeasurements are involved.\n3.  **Purpose:** Time series analysis involves methods for extracting meaningful \nstatistics and characteristics from time series data, and time series forecasting \nuses models to predict future values based on past observations.\n\n  - expected output: None\n  - context: None\n  - retrieval context: None\n\n======================================================================\n\nOverall Metric Pass Rates\n\nTask Completion: 100.00% pass rate\n\n======================================================================\n\n\u2713 Tests finished \"\ud83c\udf89\"! Run 'deepeval login' to save and analyze evaluation results\n on Confident AI.<\/code><\/pre>\n
      Our agent passed the test with honor: 100%!<\/p>\n
      You can learn much more about the DeepEval<\/strong> library in this link [8]<\/a>.<\/p>\n
      Finally, in the next section, we will learn the capabilities of Agno\u2019s library for monitoring agents.<\/p>\n
      Agent Monitoring<\/h2>\n
      Like I told you in my previous post [9]<\/a>, I chose Agno<\/strong> to learn more about Agentic AI. Just to be clear, this is not a sponsored post. It is just that I think this is the best option for those starting their journey learning about this topic.<\/p>\n
      So, one of the cool things we can take advantage of using Agno\u2019s framework is the app they make available for model monitoring.<\/p>\n
      Take this agent that can search the internet and write Instagram posts, for example.<\/p>\n
      # Imports\nimport os\nfrom agno.agent import Agent\nfrom agno.models.google import Gemini\nfrom agno.tools.file import FileTools\nfrom agno.tools.googlesearch import GoogleSearchTools\n\n\n# Topic\ntopic = \"Healthy Eating\"\n\n# Create agent\nagent = Agent(\n    model= Gemini(id=\"gemini-1.5-flash\",\n                  api_key = os.environ.get(\"GEMINI_API_KEY\")),\n                  description= f\"\"\"You are a social media marketer specialized in creating engaging content.\n                  Search the internet for 'trending topics about {topic}' and use them to create a post.\"\"\",\n                  tools=[FileTools(save_files=True),\n                         GoogleSearchTools()],\n                  expected_output=\"\"\"A short post for instagram and a prompt for a picture related to the content of the post.\n                  Don't use emojis or special characters in the post. If you find an error in the character encoding, remove the character before saving the file.\n                  Use the template:\n                  - Post\n                  - Prompt for the picture\n                  Save the post to a file named 'post.txt'.\"\"\",\n                  show_tool_calls=True,\n                  monitoring=True)\n\n# Writing and saving a file\nagent.print_response(\"\"\"Write a short post for instagram with tips and tricks that positions me as \n                     an authority in {topic}.\"\"\",\n                     markdown=True)<\/code><\/pre>\n
      To monitor its performance, follow these steps:<\/p>\n
        \n
      1. Go to https:\/\/app.agno.com\/settings<\/a> and get an API Key. <\/li>\n
      2. Open a terminal and type ag setup<\/code>.<\/li>\n
      3. If it is the first time, it might ask for the API Key. Copy and Paste it in the terminal prompt.<\/li>\n
      4. You will see the Dashboard <\/strong>tab open in your browser.<\/li>\n
      5. If you want to monitor your agent, add the argument monitoring=True<\/code>.<\/li>\n
      6. Run your agent.<\/li>\n
      7. Go to the Dashboard on the web browser.<\/li>\n
      8. Click on Sessions<\/strong>. As it is a single agent, you will see it under the tab Agents on the top portion of the page.<\/li>\n<\/ol>\n
        \"\"
        Agno Dashboard after running the agent. Image by the author.<\/figcaption><\/figure>\n
        The cools features we can see there are:<\/p>\n