A Step-by-Step Guide<\/h4>\n![\"\"](\"https:\/\/i0.wp.com\/cdn-images-1.medium.com\/max\/1024\/1%2AKKKdVTkdU_39D7lrlw4Ugw.png?ssl=1\")
High-level Overview of an LLM Agent.<\/strong> (Image by\u00a0author)<\/figcaption><\/figure>\nWhy build a general-purpose agent? <\/strong>Because it\u2019s an excellent tool to prototype your use cases and lays the groundwork for designing your own custom agentic architecture.<\/p>\nBefore we dive in, let\u2019s quickly introduce LLM agents. Feel free to skip\u00a0ahead.<\/em><\/p>\nWhat is an LLM\u00a0agent?<\/h3>\nAn LLM agent is a program whose execution logic is controlled by its underlying model.<\/p><\/blockquote>\n![\"\"](\"https:\/\/i0.wp.com\/cdn-images-1.medium.com\/max\/1024\/1%2A3B6dghRN0Ut1TCqbvIRSUA.jpeg?ssl=1\")
From Standalone LLMs to Agentic Systems.<\/strong> (Image by\u00a0author)<\/em><\/figcaption><\/figure>\nWhat sets an LLM agent apart from approaches like few-shot prompting or fixed workflows is its ability to define and adapt the steps required to execute a user\u2019s query. Given access to a set of tools (like code execution or web search), the agent can decide which tool to use, how to use it, and iterate on results based on the output. This adaptability enables the system to handle diverse use cases with minimal configuration.<\/p>\n![\"\"](\"https:\/\/i0.wp.com\/cdn-images-1.medium.com\/max\/1024\/1%2Aqc9SCzDgBjV7qQPH4zHhxA.jpeg?ssl=1\")
A Spectrum of Agentic Architectures. <\/strong>(Image by\u00a0author)<\/figcaption><\/figure>\nAgentic architectures exist on a spectrum, ranging from the reliability of fixed workflows to the flexibility of autonomous agents. For instance, a fixed flow like Retrieval-Augmented Generation (RAG<\/a>) can be enhanced with a self-reflection loop, enabling the program to iterate when the initial response falls short. Alternatively, a ReAct<\/a> agent can be equipped with fixed flows as tools, offering a flexible yet structured approach. The choice of architecture ultimately depends on the use case and the desired trade-off between reliability and flexibility.<\/p>\nFor a deeper overview, check out this\u00a0video<\/a>.<\/p>\nLet\u2019s build a general-purpose LLM agent from\u00a0scratch!<\/h3>\nStep 1. Select the right\u00a0LLM<\/h4>\n
Choosing the right model is critical to achieving your desired performance. There are several factors to consider, like licensing, cost, and language support. The most important consideration for building an LLM agent is the model\u2019s performance on key tasks like coding, tool calling, and reasoning. Benchmarks to evaluate\u00a0include:<\/p>\n
\n- \nMassive Multitask Language Understanding (MMLU)<\/a> (reasoning)<\/li>\n
- \nBerkeley\u2019s Function Calling Leaderboard<\/a> (tool selection & tool\u00a0calling)<\/li>\n
- \nHumanEval<\/a> and BigCodeBench<\/a> (coding)<\/li>\n<\/ul>\n
Another crucial factor is the model\u2019s context window. Agentic workflows can eat up a lot of tokens\u200a\u2014\u200asometimes 100K or more\u200a\u2014\u200aa larger context window is really\u00a0helpful.<\/p>\n
Models to Consider<\/strong> (at the time of\u00a0writing)<\/p>\n\n- Frontier models: GPT4-o<\/a>, Claude\u00a03.5<\/a>\n<\/li>\n
- Open-source models: Llama3.2<\/a>, Qwen2.5<\/a>.<\/li>\n<\/ul>\n
In general, larger models tend to offer better performance, but smaller models that can run locally are still a solid option. With smaller models, you\u2019ll be limited to simpler use cases and might only be able to connect your agent to one or two basic\u00a0tools.<\/p>\n
Step 2. Define the agent\u2019s control logic (aka communication structure)<\/h4>\n![\"\"](\"https:\/\/i0.wp.com\/cdn-images-1.medium.com\/max\/1024\/1%2AnV1TFBv7PTg4GlimjpkXHg.png?ssl=1\")
Single Agent Architecture<\/strong>. (Image by\u00a0author)<\/figcaption><\/figure>\nThe main difference between a simple LLM and an agent comes down to the system\u00a0prompt<\/strong>.<\/p>\nThe system prompt<\/a>, in the context of an LLM, is a set of instructions and contextual information provided to the model before it engages with user\u00a0queries.<\/p><\/blockquote>\nThe agentic behavior expected of the LLM can be codified within the system\u00a0prompt.<\/p>\n
Here are some common agentic patterns, which can be customized to fit your\u00a0needs:<\/p>\n
\n- \nTool Use:<\/strong> The agent determines when to route queries to the appropriate tool or rely on its own knowledge.<\/li>\n
- \nReflection:<\/strong> The agent reviews and corrects its answers before responding to the user. A reflection step can also be added to most LLM\u00a0systems.<\/li>\n
- \nReason-then-Act (<\/strong>ReAct<\/strong><\/a>):<\/strong> The agent iteratively reasons through how to solve the query, performs an action, observes the outcome, and determines whether to take another action or provide a response.<\/li>\n
- \nPlan-then-Execute:<\/strong> The agent plans upfront by breaking the task into sub-steps (if needed) and then executes each\u00a0step.<\/li>\n<\/ul>\n
The last two patterns\u200a\u2014\u200aReAct<\/strong> and Plan-then-Execute<\/strong>\u200a\u2014\u200aaare often the best starting point for building a general-purpose single\u00a0agent.<\/p>\n![\"\"](\"https:\/\/i0.wp.com\/cdn-images-1.medium.com\/max\/1024\/1%2A1F63COXi6OHYRiWhN85XFA.png?ssl=1\")
Overview of Common Agentic Patterns.<\/strong> (Image by\u00a0author)<\/figcaption><\/figure>\nTo implement these behaviors effectively, you\u2019ll need to do some prompt engineering. You might also want to use a structured generation<\/strong><\/a> <\/strong>technique. This basically means shaping the LLM\u2019s output to match a specific format or schema, so the agent\u2019s responses stay consistent with the communication style you\u2019re aiming\u00a0for.<\/p>\nExample:<\/strong> Below is a system prompt excerpt for a ReAct style agent from the Bee Agent Framework<\/a>.<\/p>\n# Communication structure
You communicate only in instruction lines. The format is: \"Instruction: expected output\". You must only use these instruction lines and must not enter empty lines or anything else between instruction lines.
You must skip the instruction lines Function Name, Function Input and Function Output if no function calling is required.
Message: User's message. You never use this instruction line.
Thought: A single-line plan of how to answer the user's message. It must be immediately followed by Final Answer.
Thought: A single-line step-by-step plan of how to answer the user's message. You can use the available functions defined above. This instruction line must be immediately followed by Function Name if one of the available functions defined above needs to be called, or by Final Answer. Do not provide the answer here.
Function Name: Name of the function. This instruction line must be immediately followed by Function Input.
Function Input: Function parameters. Empty object is a valid parameter.
Function Output: Output of the function in JSON format.
Thought: Continue your thinking process.
Final Answer: Answer the user or ask for more information or clarification. It must always be preceded by Thought.
## Examples
Message: Can you translate \"How are you\" into French?
Thought: The user wants to translate a text into French. I can do that.
Final Answer: Comment vas-tu?<\/pre>\nStep 3. Define the agent\u2019s core instructions<\/h4>\n
We tend to take for granted that LLMs come with a bunch of features right out of the box. Some of these are great, but others might not be exactly what you need. To get the performance you\u2019re after, it\u2019s important to spell out all the features you want\u200a\u2014\u200aand don\u2019t want\u200a\u2014\u200ain the system\u00a0prompt.<\/p>\n
This could include instructions like:<\/p>\n
\n- \nAgent Name and Role:<\/strong> What the agent is called and what it\u2019s meant to\u00a0do.<\/li>\n
- \nTone and Conciseness:<\/strong> How formal or casual it should sound, and how brief it should\u00a0be.<\/li>\n
- \nWhen to Use Tools:<\/strong> Deciding when to rely on external tools versus the model\u2019s own knowledge.<\/li>\n
- \nHandling Errors:<\/strong> What the agent should do when something goes wrong with a tool or\u00a0process.<\/li>\n<\/ul>\n
Example:<\/strong> Below is a snippet of the instructions section from the Bee Agent Framework<\/a>.<\/p>\n# Instructions
User can only see the Final Answer, all answers must be provided there.
You must always use the communication structure and instructions defined above. Do not forget that Thought must be a single-line immediately followed by Final Answer.
You must always use the communication structure and instructions defined above. Do not forget that Thought must be a single-line immediately followed by either Function Name or Final Answer.
Functions must be used to retrieve factual or historical information to answer the message.
If the user suggests using a function that is not available, answer that the function is not available. You can suggest alternatives if appropriate.
When the message is unclear or you need more information from the user, ask in Final Answer.
# Your capabilities
Prefer to use these capabilities over functions.
- You understand these languages: English, Spanish, French.
- You can translate and summarize, even long documents.
# Notes
- If you don't know the answer, say that you don't know.
- The current time and date in ISO format can be found in the last message.
- When answering the user, use friendly formats for time and date.
- Use markdown syntax for formatting code snippets, links, JSON, tables, images, files.
- Sometimes, things don't go as planned. Functions may not provide useful information on the first few tries. You should always try a few different approaches before declaring the problem unsolvable.
- When the function doesn't give you what you were asking for, you must either use another function or a different function input.
- When using search engines, you try different formulations of the query, possibly even in a different language.
- You cannot do complex calculations, computations, or data manipulations without using functions.m<\/pre>\nStep 4. Define and optimize your core\u00a0tools<\/h4>\n
Tools are what give your agents their superpowers. With a narrow set of well-defined tools, you can achieve broad functionality. Key tools to include are code execution, web search, file reading, and data analysis.<\/p>\n
For each tool, you\u2019ll need to define the following and include it as part of the system\u00a0prompt:<\/p>\n
\n- \nTool Name:<\/strong> A unique, descriptive name for the capability.<\/li>\n
- \nTool Description:<\/strong> A clear explanation of what the tool does and when to use it. This helps the agent determine when to pick the right\u00a0tool.<\/li>\n
- \nTool Input Schema:<\/strong> A schema that outlines required and optional parameters, their types, and any constraints. The agent uses this to fill in the inputs it needs based on the user\u2019s\u00a0query..<\/li>\n
- A pointer to where\/how to run the\u00a0tool.<\/li>\n<\/ul>\n
Example:<\/strong> Below is an excerpt of an Arxiv tool implementation from Langchain Community<\/a>.<\/p>\nclass ArxivInput(BaseModel):
\"\"\"Input for the Arxiv tool.\"\"\"
query: str = Field(description=\"search query to look up\")
class ArxivQueryRun(BaseTool): # type: ignore[override, override]
\"\"\"Tool that searches the Arxiv API.\"\"\"
name: str = \"arxiv\"
description: str = (
\"A wrapper around Arxiv.org \"
\"Useful for when you need to answer questions about Physics, Mathematics, \"
\"Computer Science, Quantitative Biology, Quantitative Finance, Statistics, \"
\"Electrical Engineering, and Economics \"
\"from scientific articles on arxiv.org. \"
\"Input should be a search query.\"
)
api_wrapper: ArxivAPIWrapper = Field(default_factory=ArxivAPIWrapper) # type: ignore[arg-type]
args_schema: Type[BaseModel] = ArxivInput
def _run(
self,
query: str,
run_manager: Optional[CallbackManagerForToolRun] = None,
) -> str:
\"\"\"Use the Arxiv tool.\"\"\"
return self.api_wrapper.run(query)p<\/pre>\nIn certain cases, you\u2019ll need to optimize tools to get the performance you\u2019re looking for. This might involve tweaking the tool name or description with some prompt engineering, setting up advanced configurations to handle common errors, or filtering the tool\u2019s\u00a0output.<\/p>\n
Step 5. Decide on a memory handling\u00a0strategy<\/h4>\n
LLMs are limited by their context window\u200a\u2014\u200athe number of tokens they can \u201cremember\u201d at a time. This memory can fill up fast with things like past interactions in multi-turn conversations, lengthy tool outputs, or extra context the agent is grounded on. That\u2019s why having a solid memory handling strategy is\u00a0crucial.<\/p>\n
\nMemory,<\/strong> in the context of an agent, refers to the system\u2019s capability to store, recall, and utilize information from past interactions. This enables the agent to maintain context over time, improve its responses based on previous exchanges, and provide a more personalized experience.<\/p><\/blockquote>\nCommon Memory Handling Strategies:<\/strong><\/p>\n\n- \nSliding Memory:<\/strong> Keep the last k<\/em> conversation turns in memory and drop the older\u00a0ones.<\/li>\n
- \nToken Memory:<\/strong> Keep the last n<\/em> tokens and forget the\u00a0rest.<\/li>\n
- \nSummarized Memory:<\/strong> Use the LLM to summarize the conversation at each turn and drop the individual messages.<\/li>\n<\/ul>\n
Additionally, you can also have an LLM detect key moments to store in long-term memory. This allows the agent to \u201cremember\u201d important facts about the user, making the experience even more personalized.<\/p>\n
The five steps we\u2019ve covered so far lay the foundation for setting up an agent. But what happens if we run a user query through our LLM at this\u00a0stage?<\/p>\n![\"\"](\"https:\/\/i0.wp.com\/cdn-images-1.medium.com\/max\/1024\/1%2A4gwtct9L3Y3g61LdlLdl9A.png?ssl=1\")
Answer: you get a raw text output.<\/strong> (Image by\u00a0author)<\/figcaption><\/figure>\nHere\u2019s an example of what that might look\u00a0like:<\/p>\n
User Message: Extract key insighs from this dataset
Files: bill-of-materials.csv
Thought: First, I need to inspect the columns of the dataset and provide basic data statistics.
Function Name: Python
Function Input: {\"language\":\"python\",\"code\":\"import pandas as pdnndataset = pd.read_csv('bill-of-materials.csv')nnprint(dataset.columns)nprint(dataset.describe())\",\"inputFiles\":[\"bill-of-materials.csv\"]}<\/pre>\nAt this point, the agent produces raw text output. So how do we get it to actually execute the next step? That\u2019s where parsing and orchestration come\u00a0in.<\/p>\n
Step 6. Parse the agent\u2019s raw\u00a0output<\/strong><\/h4>\nA parser<\/strong> is a function that converts raw data into a format your application can understand and work with (like an object with properties)<\/p><\/blockquote>\nFor the agent we\u2019re building, the parser needs to recognize the communication structure we defined in Step 2<\/strong> and return a structured output, like JSON. This makes it easier for the application to process and execute the agent\u2019s next\u00a0steps.<\/p>\nNote: some model providers like <\/em>OpenAI<\/em><\/a>, can return parsable outputs by default. For other models, especially open-source ones, this would need to be configured.<\/em><\/p>\nStep 7. Orchestrate the agent\u2019s next\u00a0step<\/h4>\n
The final step is setting up the orchestration logic. This determines what happens after the LLM outputs a result. Depending on the output, you\u2019ll\u00a0either:<\/p>\n
\n- \nExecute a tool call<\/strong>,\u00a0or<\/li>\n
- \nReturn an answer<\/strong>\u200a\u2014\u200aeither the final response to the user\u2019s query or a follow-up request for more information.<\/li>\n<\/ol>\n
![\"\"](\"https:\/\/i0.wp.com\/cdn-images-1.medium.com\/max\/1024\/1%2AsBpr4BfqAgpyQ57GkP3Ubw.png?ssl=1\")
Extended single agent architecture.<\/strong> (Image by\u00a0author)<\/figcaption><\/figure>\nIf a tool call is triggered, the tool\u2019s output is sent back to the LLM (as part of its working memory). The LLM would then determine what to do with this new information: either performan another tool call or return an answer to the\u00a0user.<\/p>\n
Here\u2019s an example of how this orchestration logic might look in\u00a0code:<\/p>\n
def orchestrator(llm_agent, llm_output, tools, user_query):
\"\"\"
Orchestrates the response based on LLM output and iterates if necessary.
Parameters:
- llm_agent (callable): The LLM agent function for processing tool outputs.
- llm_output (dict): Initial output from the LLM, specifying the next action.
- tools (dict): Dictionary of available tools with their execution methods.
- user_query (str): The original user query.
Returns:
- str: The final response to the user.
\"\"\"
while True:
action = llm_output.get(\"action\")
if action == \"tool_call\":
# Extract tool name and parameters
tool_name = llm_output.get(\"tool_name\")
tool_params = llm_output.get(\"tool_params\", {})
if tool_name in tools:
try:
# Execute the tool
tool_result = tools[tool_name](**tool_params)
# Send tool output back to the LLM agent for further processing
llm_output = llm_agent({\"tool_output\": tool_result})
except Exception as e:
return f\"Error executing tool '{tool_name}': {str(e)}\"
else:
return f\"Error: Tool '{tool_name}' not found.\"
elif action == \"return_answer\":
# Return the final answer to the user
return llm_output.get(\"answer\", \"No answer provided.\")
else:
return \"Error: Unrecognized action type from LLM output.\"<\/pre>\nAnd voil\u00e0! <\/strong>You now have a system capable of handling a wide variety of use cases\u200a\u2014\u200afrom competitive analysis and advanced research to automating complex workflows.<\/p>\nWhere do multi-agent systems come\u00a0in?<\/h3>\n
While this generation of LLMs is incredibly powerful, they have a key limitation: they struggle with information overload<\/a>. Too much context or too many tools can overwhelm the model, leading to performance issues. A general-purpose single agent will eventually hit this ceiling, especially since agents are notoriously token-hungry.<\/p>\nFor certain use cases, a multi-agent setup might make more sense. By dividing responsibilities across multiple agents, you can avoid overloading the context of a single LLM agent and improve overall efficiency.<\/p>\n
That said, a general-purpose single-agent setup is a fantastic starting point for prototyping<\/strong>. It can help you quickly test your use case and identify where things start to break down. Through this process, you\u00a0can:<\/p>\n\n- Understand which parts of the task truly benefit from an agentic approach.<\/li>\n
- Identify components that can be spun off as standalone processes in a larger workflow.<\/li>\n<\/ol>\n
Starting with a single agent gives you valuable insights to refine your approach as you scale to more complex\u00a0systems.<\/p>\n
What is the best way to get\u00a0started?<\/h3>\n
Ready to dive in and start building? Using a framework can be a great way to quickly test and iterate on your agent configuration.<\/p>\n
\n- \nPlanning on using frontier models like OpenAI?<\/strong> Try this tutorial<\/a> from LangGraph<\/a>.<\/li>\n
- \nPlanning on using open-source models like Llama 3.2?<\/strong> Try this starter template<\/a> from the Bee Agent Framework<\/a>.<\/li>\n<\/ul>\n
What\u2019s your experience building general-purpose agents?
Share your in the comments!<\/em><\/p>\n![\"\"](\"https:\/\/medium.com\/_\/stat?event=post.clientViewed&referrerSource=full_rss&postId=c40be49e7400\")
<\/p>\n
\nHow to Build a General-Purpose LLM Agent<\/a> was originally published in Towards Data Science<\/a> on Medium, where people are continuing the conversation by highlighting and responding to this story.<\/p>\n<\/div>\n
Why build a general-purpose agent? <\/strong>Because it\u2019s an excellent tool to prototype your use cases and lays the groundwork for designing your own custom agentic architecture.<\/p>\n Before we dive in, let\u2019s quickly introduce LLM agents. Feel free to skip\u00a0ahead.<\/em><\/p>\n An LLM agent is a program whose execution logic is controlled by its underlying model.<\/p><\/blockquote>\n What sets an LLM agent apart from approaches like few-shot prompting or fixed workflows is its ability to define and adapt the steps required to execute a user\u2019s query. Given access to a set of tools (like code execution or web search), the agent can decide which tool to use, how to use it, and iterate on results based on the output. This adaptability enables the system to handle diverse use cases with minimal configuration.<\/p>\n Agentic architectures exist on a spectrum, ranging from the reliability of fixed workflows to the flexibility of autonomous agents. For instance, a fixed flow like Retrieval-Augmented Generation (RAG<\/a>) can be enhanced with a self-reflection loop, enabling the program to iterate when the initial response falls short. Alternatively, a ReAct<\/a> agent can be equipped with fixed flows as tools, offering a flexible yet structured approach. The choice of architecture ultimately depends on the use case and the desired trade-off between reliability and flexibility.<\/p>\n For a deeper overview, check out this\u00a0video<\/a>.<\/p>\n Choosing the right model is critical to achieving your desired performance. There are several factors to consider, like licensing, cost, and language support. The most important consideration for building an LLM agent is the model\u2019s performance on key tasks like coding, tool calling, and reasoning. Benchmarks to evaluate\u00a0include:<\/p>\n Another crucial factor is the model\u2019s context window. Agentic workflows can eat up a lot of tokens\u200a\u2014\u200asometimes 100K or more\u200a\u2014\u200aa larger context window is really\u00a0helpful.<\/p>\n Models to Consider<\/strong> (at the time of\u00a0writing)<\/p>\n In general, larger models tend to offer better performance, but smaller models that can run locally are still a solid option. With smaller models, you\u2019ll be limited to simpler use cases and might only be able to connect your agent to one or two basic\u00a0tools.<\/p>\n The main difference between a simple LLM and an agent comes down to the system\u00a0prompt<\/strong>.<\/p>\n The system prompt<\/a>, in the context of an LLM, is a set of instructions and contextual information provided to the model before it engages with user\u00a0queries.<\/p><\/blockquote>\n The agentic behavior expected of the LLM can be codified within the system\u00a0prompt.<\/p>\n Here are some common agentic patterns, which can be customized to fit your\u00a0needs:<\/p>\n The last two patterns\u200a\u2014\u200aReAct<\/strong> and Plan-then-Execute<\/strong>\u200a\u2014\u200aaare often the best starting point for building a general-purpose single\u00a0agent.<\/p>\n To implement these behaviors effectively, you\u2019ll need to do some prompt engineering. You might also want to use a structured generation<\/strong><\/a> <\/strong>technique. This basically means shaping the LLM\u2019s output to match a specific format or schema, so the agent\u2019s responses stay consistent with the communication style you\u2019re aiming\u00a0for.<\/p>\n Example:<\/strong> Below is a system prompt excerpt for a ReAct style agent from the Bee Agent Framework<\/a>.<\/p>\n We tend to take for granted that LLMs come with a bunch of features right out of the box. Some of these are great, but others might not be exactly what you need. To get the performance you\u2019re after, it\u2019s important to spell out all the features you want\u200a\u2014\u200aand don\u2019t want\u200a\u2014\u200ain the system\u00a0prompt.<\/p>\n This could include instructions like:<\/p>\n Example:<\/strong> Below is a snippet of the instructions section from the Bee Agent Framework<\/a>.<\/p>\n Tools are what give your agents their superpowers. With a narrow set of well-defined tools, you can achieve broad functionality. Key tools to include are code execution, web search, file reading, and data analysis.<\/p>\n For each tool, you\u2019ll need to define the following and include it as part of the system\u00a0prompt:<\/p>\n Example:<\/strong> Below is an excerpt of an Arxiv tool implementation from Langchain Community<\/a>.<\/p>\n In certain cases, you\u2019ll need to optimize tools to get the performance you\u2019re looking for. This might involve tweaking the tool name or description with some prompt engineering, setting up advanced configurations to handle common errors, or filtering the tool\u2019s\u00a0output.<\/p>\n LLMs are limited by their context window\u200a\u2014\u200athe number of tokens they can \u201cremember\u201d at a time. This memory can fill up fast with things like past interactions in multi-turn conversations, lengthy tool outputs, or extra context the agent is grounded on. That\u2019s why having a solid memory handling strategy is\u00a0crucial.<\/p>\n \nMemory,<\/strong> in the context of an agent, refers to the system\u2019s capability to store, recall, and utilize information from past interactions. This enables the agent to maintain context over time, improve its responses based on previous exchanges, and provide a more personalized experience.<\/p><\/blockquote>\n Common Memory Handling Strategies:<\/strong><\/p>\n Additionally, you can also have an LLM detect key moments to store in long-term memory. This allows the agent to \u201cremember\u201d important facts about the user, making the experience even more personalized.<\/p>\n The five steps we\u2019ve covered so far lay the foundation for setting up an agent. But what happens if we run a user query through our LLM at this\u00a0stage?<\/p>\n Here\u2019s an example of what that might look\u00a0like:<\/p>\n At this point, the agent produces raw text output. So how do we get it to actually execute the next step? That\u2019s where parsing and orchestration come\u00a0in.<\/p>\n A parser<\/strong> is a function that converts raw data into a format your application can understand and work with (like an object with properties)<\/p><\/blockquote>\n For the agent we\u2019re building, the parser needs to recognize the communication structure we defined in Step 2<\/strong> and return a structured output, like JSON. This makes it easier for the application to process and execute the agent\u2019s next\u00a0steps.<\/p>\n Note: some model providers like <\/em>OpenAI<\/em><\/a>, can return parsable outputs by default. For other models, especially open-source ones, this would need to be configured.<\/em><\/p>\n The final step is setting up the orchestration logic. This determines what happens after the LLM outputs a result. Depending on the output, you\u2019ll\u00a0either:<\/p>\n If a tool call is triggered, the tool\u2019s output is sent back to the LLM (as part of its working memory). The LLM would then determine what to do with this new information: either performan another tool call or return an answer to the\u00a0user.<\/p>\n Here\u2019s an example of how this orchestration logic might look in\u00a0code:<\/p>\n And voil\u00e0! <\/strong>You now have a system capable of handling a wide variety of use cases\u200a\u2014\u200afrom competitive analysis and advanced research to automating complex workflows.<\/p>\n While this generation of LLMs is incredibly powerful, they have a key limitation: they struggle with information overload<\/a>. Too much context or too many tools can overwhelm the model, leading to performance issues. A general-purpose single agent will eventually hit this ceiling, especially since agents are notoriously token-hungry.<\/p>\n For certain use cases, a multi-agent setup might make more sense. By dividing responsibilities across multiple agents, you can avoid overloading the context of a single LLM agent and improve overall efficiency.<\/p>\n That said, a general-purpose single-agent setup is a fantastic starting point for prototyping<\/strong>. It can help you quickly test your use case and identify where things start to break down. Through this process, you\u00a0can:<\/p>\n Starting with a single agent gives you valuable insights to refine your approach as you scale to more complex\u00a0systems.<\/p>\n Ready to dive in and start building? Using a framework can be a great way to quickly test and iterate on your agent configuration.<\/p>\n What\u2019s your experience building general-purpose agents? How to Build a General-Purpose LLM Agent<\/a> was originally published in Towards Data Science<\/a> on Medium, where people are continuing the conversation by highlighting and responding to this story.<\/p>\n<\/div>\nWhat is an LLM\u00a0agent?<\/h3>\n
Let\u2019s build a general-purpose LLM agent from\u00a0scratch!<\/h3>\n
Step 1. Select the right\u00a0LLM<\/h4>\n
\n
\n
Step 2. Define the agent\u2019s control logic (aka communication structure)<\/h4>\n
\n
# Communication structure
You communicate only in instruction lines. The format is: \"Instruction: expected output\". You must only use these instruction lines and must not enter empty lines or anything else between instruction lines.
You must skip the instruction lines Function Name, Function Input and Function Output if no function calling is required.
Message: User's message. You never use this instruction line.
Thought: A single-line plan of how to answer the user's message. It must be immediately followed by Final Answer.
Thought: A single-line step-by-step plan of how to answer the user's message. You can use the available functions defined above. This instruction line must be immediately followed by Function Name if one of the available functions defined above needs to be called, or by Final Answer. Do not provide the answer here.
Function Name: Name of the function. This instruction line must be immediately followed by Function Input.
Function Input: Function parameters. Empty object is a valid parameter.
Function Output: Output of the function in JSON format.
Thought: Continue your thinking process.
Final Answer: Answer the user or ask for more information or clarification. It must always be preceded by Thought.
## Examples
Message: Can you translate \"How are you\" into French?
Thought: The user wants to translate a text into French. I can do that.
Final Answer: Comment vas-tu?<\/pre>\nStep 3. Define the agent\u2019s core instructions<\/h4>\n
\n
# Instructions
User can only see the Final Answer, all answers must be provided there.
You must always use the communication structure and instructions defined above. Do not forget that Thought must be a single-line immediately followed by Final Answer.
You must always use the communication structure and instructions defined above. Do not forget that Thought must be a single-line immediately followed by either Function Name or Final Answer.
Functions must be used to retrieve factual or historical information to answer the message.
If the user suggests using a function that is not available, answer that the function is not available. You can suggest alternatives if appropriate.
When the message is unclear or you need more information from the user, ask in Final Answer.
# Your capabilities
Prefer to use these capabilities over functions.
- You understand these languages: English, Spanish, French.
- You can translate and summarize, even long documents.
# Notes
- If you don't know the answer, say that you don't know.
- The current time and date in ISO format can be found in the last message.
- When answering the user, use friendly formats for time and date.
- Use markdown syntax for formatting code snippets, links, JSON, tables, images, files.
- Sometimes, things don't go as planned. Functions may not provide useful information on the first few tries. You should always try a few different approaches before declaring the problem unsolvable.
- When the function doesn't give you what you were asking for, you must either use another function or a different function input.
- When using search engines, you try different formulations of the query, possibly even in a different language.
- You cannot do complex calculations, computations, or data manipulations without using functions.m<\/pre>\nStep 4. Define and optimize your core\u00a0tools<\/h4>\n
\n
class ArxivInput(BaseModel):
\"\"\"Input for the Arxiv tool.\"\"\"
query: str = Field(description=\"search query to look up\")
class ArxivQueryRun(BaseTool): # type: ignore[override, override]
\"\"\"Tool that searches the Arxiv API.\"\"\"
name: str = \"arxiv\"
description: str = (
\"A wrapper around Arxiv.org \"
\"Useful for when you need to answer questions about Physics, Mathematics, \"
\"Computer Science, Quantitative Biology, Quantitative Finance, Statistics, \"
\"Electrical Engineering, and Economics \"
\"from scientific articles on arxiv.org. \"
\"Input should be a search query.\"
)
api_wrapper: ArxivAPIWrapper = Field(default_factory=ArxivAPIWrapper) # type: ignore[arg-type]
args_schema: Type[BaseModel] = ArxivInput
def _run(
self,
query: str,
run_manager: Optional[CallbackManagerForToolRun] = None,
) -> str:
\"\"\"Use the Arxiv tool.\"\"\"
return self.api_wrapper.run(query)p<\/pre>\nStep 5. Decide on a memory handling\u00a0strategy<\/h4>\n
\n
User Message: Extract key insighs from this dataset
Files: bill-of-materials.csv
Thought: First, I need to inspect the columns of the dataset and provide basic data statistics.
Function Name: Python
Function Input: {\"language\":\"python\",\"code\":\"import pandas as pdnndataset = pd.read_csv('bill-of-materials.csv')nnprint(dataset.columns)nprint(dataset.describe())\",\"inputFiles\":[\"bill-of-materials.csv\"]}<\/pre>\nStep 6. Parse the agent\u2019s raw\u00a0output<\/strong><\/h4>\n
Step 7. Orchestrate the agent\u2019s next\u00a0step<\/h4>\n
\n
def orchestrator(llm_agent, llm_output, tools, user_query):
\"\"\"
Orchestrates the response based on LLM output and iterates if necessary.
Parameters:
- llm_agent (callable): The LLM agent function for processing tool outputs.
- llm_output (dict): Initial output from the LLM, specifying the next action.
- tools (dict): Dictionary of available tools with their execution methods.
- user_query (str): The original user query.
Returns:
- str: The final response to the user.
\"\"\"
while True:
action = llm_output.get(\"action\")
if action == \"tool_call\":
# Extract tool name and parameters
tool_name = llm_output.get(\"tool_name\")
tool_params = llm_output.get(\"tool_params\", {})
if tool_name in tools:
try:
# Execute the tool
tool_result = tools[tool_name](**tool_params)
# Send tool output back to the LLM agent for further processing
llm_output = llm_agent({\"tool_output\": tool_result})
except Exception as e:
return f\"Error executing tool '{tool_name}': {str(e)}\"
else:
return f\"Error: Tool '{tool_name}' not found.\"
elif action == \"return_answer\":
# Return the final answer to the user
return llm_output.get(\"answer\", \"No answer provided.\")
else:
return \"Error: Unrecognized action type from LLM output.\"<\/pre>\nWhere do multi-agent systems come\u00a0in?<\/h3>\n
\n
What is the best way to get\u00a0started?<\/h3>\n
\n
Share your in the comments!<\/em><\/p>\n<\/p>\n
\n