Introduction<\/mdspan><\/h2>\nI\u2019ve always<\/mdspan> been fascinated by debates\u2014the strategic framing, the sharp retorts, and the carefully timed comebacks. Debates aren\u2019t just entertaining; they\u2019re structured battles of ideas, driven by logic and evidence. Recently, I started wondering: could we replicate that dynamic using AI agents\u2014having them debate each other autonomously, complete with real-time fact-checking and moderation? The result was Deb8flow<\/strong>, an autonomous AI debating environment powered by LangGraph<\/a><\/strong>, OpenAI\u2019s GPT-4o<\/strong> model, and the new integrated Web Search<\/a><\/strong> feature.<\/p>\nIn Deb8flow, two agents\u2014Pro and Con\u2014square off on a given topic while a Moderator manages turn-taking. A dedicated Fact Checker reviews every claim in real time using GPT-4o\u2019s new browsing capabilities, and a final Judge evaluates the arguments for quality and coherence. If an agent repeatedly makes factual errors, they\u2019re automatically disqualified\u2014ensuring the debate stays grounded in truth.<\/p>\n
This article offers an in-depth look at the advanced architecture and dynamic workflows that power autonomous AI debates. I\u2019ll walk you through how Deb8flow\u2019s modular design leverages LangGraph\u2019s state management and conditional routing, alongside GPT-4o\u2019s capabilities.<\/p>\n
Even if you\u2019re new to AI agents or LangGraph (see resources [1] and [2] for primers), I\u2019ll explain the key concepts clearly. And if you\u2019d like to explore further, the full project is available on GitHub: iason-solomos\/Deb8flow<\/strong><\/a>.<\/p>\nReady to see how AI agents can debate autonomously in practice?<\/p>\n
Let\u2019s dive in.<\/strong><\/p>\nHigh-Level Overview: Autonomous Debates with Multiple Agents<\/h2>\n
In Deb8flow, we orchestrate a formal debate<\/strong> between two AI agents \u2013 one arguing Pro<\/strong> and one Con<\/strong> \u2013 complete with a Moderator<\/strong>, a Fact Checker<\/strong>, and a final Judge<\/strong>. The debate unfolds autonomously, with each agent playing a role in a structured format.<\/p>\nAt its core, Deb8flow is a LangGraph-powered agent system, built atop LangChain, using GPT-4o to power each role\u2014Pro, Con, Judge, and beyond. We use GPT-4o\u2019s preview model with browsing capabilities to enable real-time fact-checking. In essence, the Pro and Con agents debate; after each statement, a fact-checker agent uses GPT-4o\u2019s web search to catch any hallucinations or inaccuracies in that statement in real time<\/strong>.\u200b The debate only continues once the statement is verified. The whole process is coordinated by a LangGraph-defined workflow that ensures proper turn-taking and conditional logic.<\/p>\n\n![\"\"](\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/04\/workflow-diagram.png?ssl=1\")
![\"\"](\"blob:https:\/\/chatgpt.com\/7bfbd09f-e5a6-4eb8-ac9a-848211727d0c\")
High-level debate flow graph. Each rectangle is an agent node (Pro\/Con debaters, Fact Checker, Judge, etc.), and diamonds are control nodes (Moderator and a router after fact-checking). Solid arrows denote the normal progression, while dashed arrows indicate retries if a claim fails fact-check. The Judge node outputs the final verdict, then the workflow ends.<\/em>
Image generated by the author with DALL-E<\/figcaption><\/figure>\n The debate workflow goes through these stages:<\/p>\n
\n- \nTopic Generation:<\/strong> A Topic Generator agent produces a nuanced, debatable topic for the session (e.g. \u201cShould AI be used in classroom education?\u201d<\/em>).<\/li>\n
- \nOpening: <\/strong>The Pro Argument <\/strong>Agent makes an opening statement in favor of the topic, kicking off the debate.<\/li>\n
- \nRebuttal: <\/strong>The Debate Moderator<\/strong> then gives the floor to the Con Argument<\/strong> agent, who rebuts the Pro\u2019s opening statement.<\/li>\n
- \nCounter: <\/strong>The Moderator gives the floor back to the Pro<\/strong> agent, who counters the Con agent\u2019s points.<\/li>\n
- \nClosing: <\/strong>The Moderator switches the floor to the Con<\/strong> agent one last time for a closing argument.<\/li>\n
- \nJudgment<\/strong>: Finally, the Judge<\/strong> agent reviews the full debate history and evaluates both sides based on argument quality, clarity, and persuasiveness. The most convincing side wins.<\/li>\n<\/ul>\n
After every single speech<\/strong>, the Fact Checker<\/strong> agent steps in to verify the factual accuracy of that statement\u200b. If a debater\u2019s claim doesn\u2019t hold up (e.g. cites a wrong statistic or \u201challucinates\u201d a fact), the workflow triggers a retry<\/em>: the speaker has to correct or modify their statement. (If either debater accumulates 3 fact-check failures, they are automatically disqualified<\/strong> for repeatedly spreading inaccuracies, and their opponent wins by default.) This mechanism keeps our AI debaters honest and grounded in reality!<\/p>\nPrerequisites and Setup<\/h2>\n
Before diving into the code, make sure you have the following in place:<\/p>\n
\n- \nPython 3.12+<\/strong> installed.<\/li>\n
- An OpenAI API key<\/strong> with access to the GPT-4o model. You can create your own API key here: https:\/\/platform.openai.com\/settings\/organization\/api-keys<\/a>\n<\/li>\n
- \nProject Code<\/strong>: Clone the Deb8flow repository from GitHub (
git clone https:\/\/github.com\/iason-solomos\/Deb8flow.git<\/code>). The repo includes a requirements.txt<\/code> for all required packages. Key dependencies include LangChain\/LangGraph (for building the agent graph) and the OpenAI Python client.<\/li>\n- \nInstall Dependencies<\/strong>: In your project directory, run:
pip install -r requirements.txt<\/code> to install the necessary libraries.<\/li>\n- Create a
.env<\/code> file in the project root to hold your OpenAI API credentials. It should be of the form: OPENAI_API_KEY_GPT4O = \"sk-\u2026\"<\/code>\n<\/li>\n- You can also at any time check out the README file: https:\/\/github.com\/iason-solomos\/Deb8flow<\/a> if you simply want to run the finished app.<\/li>\n<\/ul>\n
Once dependencies are installed and the environment variable is set, you should be ready to run the app. The project structure is organized for clarity:<\/p>\n
Deb8flow\/
\u251c\u2500\u2500 configurations\/
\u2502 \u251c\u2500\u2500 debate_constants.py
\u2502 \u2514\u2500\u2500 llm_config.py
\u251c\u2500\u2500 nodes\/
\u2502 \u251c\u2500\u2500 base_component.py
\u2502 \u251c\u2500\u2500 topic_generator_node.py
\u2502 \u251c\u2500\u2500 pro_debater_node.py
\u2502 \u251c\u2500\u2500 con_debater_node.py
\u2502 \u251c\u2500\u2500 debate_moderator_node.py
\u2502 \u251c\u2500\u2500 fact_checker_node.py
\u2502 \u251c\u2500\u2500 fact_check_router_node.py
\u2502 \u2514\u2500\u2500 judge_node.py
\u251c\u2500\u2500 prompts\/
\u2502 \u251c\u2500\u2500 topic_generator_prompts.py
\u2502 \u251c\u2500\u2500 pro_debater_prompts.py
\u2502 \u251c\u2500\u2500 con_debater_prompts.py
\u2502 \u2514\u2500\u2500 \u2026 (prompts for other agents)
\u251c\u2500\u2500 tests\/ (contains unit and whole workflow tests)
\u2514\u2500\u2500 debate_workflow.py<\/p>\n
A quick tour of this structure:<\/p>\n
configurations\/<\/code><\/strong> holds constant definitions and LLM configuration classes.<\/p>\nnodes\/<\/code><\/strong> contains the implementation of each agent or functional node in the debate (each of these is a module defining one agent\u2019s behavior).<\/p>\nprompts\/<\/code><\/strong> stores the prompt templates for the language model (so each agent knows how to prompt GPT-4o for its specific task).<\/p>\ndebate_workflow.py<\/code><\/strong> ties everything together by defining the LangGraph workflow (the graph of nodes and transitions).<\/p>\ndebate_state.py<\/code><\/strong> defines the shared data structure that the agents will be using on each run.<\/p>\ntests\/<\/code><\/strong> includes some basic tests and example runs to help you verify everything is working.<\/p>\nUnder the Hood: State Management and Workflow Setup<\/h2>\n
To coordinate a complex multi-turn debate, we need a shared state and a well-defined flow. We\u2019ll start by looking at how Deb8flow defines the debate state<\/strong> and constants, and then see how the LangGraph workflow<\/strong> is constructed.<\/p>\nDefining the Debate State Schema (debate_state.py<\/code>)<\/h3>\nDeb8flow uses a shared state<\/strong> (https:\/\/langchain-ai.github.io\/langgraph\/concepts\/low_level\/#state <\/a>) in the form of a Python TypedDict<\/code> that all agents can read from and update. This state tracks the debate\u2019s progress and context \u2013 things like the topic, the history of messages, whose turn it is, etc. By centralizing this information, each agent node can make decisions based on the current state of the debate.<\/p>\nLink: debate_state.py<\/strong><\/a><\/p>\nfrom typing import TypedDict, List, Dict, Literal\n\n\nDebateStage = Literal[\"opening\", \"rebuttal\", \"counter\", \"final_argument\"]\n\nclass DebateMessage(TypedDict):\n speaker: str # e.g. pro or con\n content: str # The message each speaker produced\n validated: bool # Whether the FactChecker ok\u2019d this message\n stage: DebateStage # The stage of the debate when this message was produced\n\nclass DebateState(TypedDict):\n debate_topic: str\n positions: Dict[str, str]\n messages: List[DebateMessage]\n opening_statement_pro_agent: str\n stage: str # \"opening\", \"rebuttal\", \"counter\", \"final_argument\"\n speaker: str # \"pro\" or \"con\"\n times_pro_fact_checked: int # The number of times the pro agent has been fact-checked. If it reaches 3, the pro agent is disqualified.\n times_con_fact_checked: int # The number of times the con agent has been fact-checked. If it reaches 3, the con agent is disqualified.<\/code><\/pre>\nKey fields that we need to have in the DebateState<\/code> include:<\/p>\n\n- \n
debate_topic<\/code> (str): The topic being debated.<\/li>\n- \n
messages<\/code> (List[DebateMessage]): A list of all messages exchanged so far. Each message is a dictionary with fields for speaker<\/code> (e.g. \"pro\"<\/code> or \"con\"<\/code> or \"fact_checker\"<\/code>), the message content<\/code> (text), a validated<\/code> flag (whether it passed fact-check), and the stage<\/code> of the debate when it was produced.<\/li>\n- \n
stage<\/code> (str): The current debate stage (one of \"opening\"<\/code>, \"rebuttal\"<\/code>, \"counter\"<\/code>, \"final_argument\"<\/code>).<\/li>\n- \n
speaker<\/code> (str): Whose turn it is currently (\"pro\"<\/code> or \"con\"<\/code>).<\/li>\n- \n
times_pro_fact_checked<\/code> \/ times_con_fact_checked<\/code> (int): Counters for how many times each side has been caught with a false claim. (In our rules, if a debater fails fact-check 3 times, they could be disqualified or automatically lose.)<\/li>\n- \n
positions<\/code> (Dict[str, str]): (Optional) A mapping of each side\u2019s general stance (e.g., \"pro\": \"In favor of the topic\"<\/code>).<\/li>\n<\/ul>\nBy structuring the debate\u2019s state, agents find it easy to access the conversation history or check the current stage, and the control logic can update the state between turns. The state is essentially the memory of the debate.<\/p>\n
Constants and Configuration<\/h3>\n
To avoid \u201cmagic strings\u201d scattered in the code, we define some constants in debate_constants.py<\/code>. For example, constants for stage names (STAGE_OPENING = \"opening\"<\/code>, etc.), speaker identifiers (SPEAKER_PRO = \"pro\"<\/code>, SPEAKER_CON = \"con\"<\/code>, etc.), and node names (NODE_PRO_DEBATER = \"pro_debater_node\"<\/code>, etc.). These make the code easier to maintain and read.<\/p>\ndebate_constants.py<\/strong><\/a>:<\/strong><\/p>\n# Stage names\nSTAGE_OPENING = \"opening\"\nSTAGE_REBUTTAL = \"rebuttal\"\nSTAGE_COUNTER = \"counter\"\nSTAGE_FINAL_ARGUMENT = \"final_argument\"\nSTAGE_END = \"end\"\n\n# Speakers\nSPEAKER_PRO = \"pro\"\nSPEAKER_CON = \"con\"\nSPEAKER_JUDGE = \"judge\"\n\n# Node names\nNODE_PRO_DEBATER = \"pro_debater_node\"\nNODE_CON_DEBATER = \"con_debater_node\"\nNODE_DEBATE_MODERATOR = \"debate_moderator_node\"\nNODE_JUDGE = \"judge_node\"\n<\/code><\/pre>\nWe also set up LLM configuration in llm_config.py<\/a>. Here, we define classes for OpenAI or Azure OpenAI configs and then create a dictionary llm_config_map<\/code> mapping model names to their config. For instance, we map \"gpt-4o\"<\/code> to an OpenAILLMConfig<\/code> that holds the model name and API key. This way, whenever we need to initialize a GPT-4o agent, we can just do llm_config_map[\"gpt-4o\"]<\/code> to get the right config. All our main agents (debaters, topic generator, judge) use this same GPT-4o configuration.<\/p>\nimport os\nfrom dataclasses import dataclass\nfrom typing import Union\n\n@dataclass\nclass OpenAILLMConfig:\n \"\"\"\n A data class to store configuration details for OpenAI models.\n\n Attributes:\n model_name (str): The name of the OpenAI model to use.\n openai_api_key (str): The API key for authenticating with the OpenAI service.\n \"\"\"\n model_name: str\n openai_api_key: str\n\n\nllm_config_map = {\n \"gpt-4o\": OpenAILLMConfig(\n model_name=\"gpt-4o\",\n openai_api_key=os.getenv(\"OPENAI_API_KEY_GPT4O\"),\n )\n}\n<\/code><\/pre>\nBuilding the LangGraph Workflow (debate_workflow.py<\/code>)<\/h3>\nWith state and configs in place, we construct the debate workflow graph<\/a>. LangGraph\u2019s StateGraph<\/strong> is the backbone that connects all our agent nodes in the order they should execute. Here\u2019s how we set it up:<\/p>\nclass DebateWorkflow:\n\n def _initialize_workflow(self) -> StateGraph:\n workflow = StateGraph(DebateState)\n # Nodes\n workflow.add_node(\"generate_topic_node\", GenerateTopicNode(llm_config_map[\"gpt-4o\"]))\n workflow.add_node(\"pro_debater_node\", ProDebaterNode(llm_config_map[\"gpt-4o\"]))\n workflow.add_node(\"con_debater_node\", ConDebaterNode(llm_config_map[\"gpt-4o\"]))\n workflow.add_node(\"fact_check_node\", FactCheckNode())\n workflow.add_node(\"fact_check_router_node\", FactCheckRouterNode())\n workflow.add_node(\"debate_moderator_node\", DebateModeratorNode())\n workflow.add_node(\"judge_node\", JudgeNode(llm_config_map[\"gpt-4o\"]))\n\n # Entry point\n workflow.set_entry_point(\"generate_topic_node\")\n\n # Flow\n workflow.add_edge(\"generate_topic_node\", \"pro_debater_node\")\n workflow.add_edge(\"pro_debater_node\", \"fact_check_node\")\n workflow.add_edge(\"con_debater_node\", \"fact_check_node\")\n workflow.add_edge(\"fact_check_node\", \"fact_check_router_node\")\n workflow.add_edge(\"judge_node\", END)\n return workflow\n\n\n\n async def run(self):\n workflow = self._initialize_workflow()\n graph = workflow.compile()\n # graph.get_graph().draw_mermaid_png(output_file_path=\"workflow_graph.png\")\n initial_state = {\n \"topic\": \"\",\n \"positions\": {}\n }\n final_state = await graph.ainvoke(initial_state, config={\"recursion_limit\": 50})\n return final_state\n<\/code><\/pre>\nLet\u2019s break down what\u2019s happening:<\/p>\n
\n- We initialize a new
StateGraph<\/code> with our DebateState<\/code> type as the state schema.<\/li>\n- We add each node (agent) to the graph with a name. For nodes that need an LLM, we pass in the GPT-4o config. For example,
\"pro_debater_node\"<\/code> is added as ProDebaterNode(llm_config_map[\"gpt-4o\"])<\/code>, meaning the Pro debater agent will use GPT-4o as its underlying model.<\/li>\n- We set the entry point<\/strong> of the graph to
\"generate_topic_node\"<\/code>. This means the first step of the workflow is to generate a debate topic.<\/li>\n- Then we add directed edges to connect nodes. The edges above encode the primary sequence: topic -> pro\u2019s turn -> fact-check -> (then a routing decision) -> \u2026 eventually -> judge -> END. We don\u2019t connect the Moderator or Fact Check Router with static edges, since these nodes use dynamic commands to redirect the flow. The final edge connects the judge to an
END<\/code> marker to terminate the graph.<\/li>\n<\/ul>\nWhen the workflow runs, control will pass along these edges in order, but whenever we hit a router or moderator node<\/strong>, that node will output a command telling the graph which node to go to next (overriding the default edge). This is how we create conditional loops: the fact_check_router_node<\/code> might send us back to a debater node for a retry, instead of following a straight line. LangGraph supports this by allowing nodes to return a special Command<\/code> object with goto<\/code> instructions.<\/p>\nIn summary, at a high level we\u2019ve defined an agentic workflow<\/strong>: a graph of autonomous agents where control can branch and loop based on the agents\u2019 outputs. Now, let\u2019s explore what each of these agent nodes actually does.<\/p>\nAgent Nodes Breakdown<\/h2>\n
Each stage or role in the debate is encapsulated in a node (agent). In LangGraph, nodes are often simple functions, but I wanted a more object-oriented approach for clarity and reusability. So in Deb8flow, every node is a class<\/strong> with a __call__<\/code> method. All the main agent classes inherit from a common BaseComponent<\/code> for shared functionality. This design makes the system modular: we can easily swap out or extend agents by modifying their class definitions, and each agent class is responsible for its piece of the workflow.<\/p>\nLet\u2019s go through the key agents one by one.<\/p>\n
\nBaseComponent<\/code> \u2013 A Reusable Agent Base Class<\/h3>\nMost of our agent nodes (like the debaters and judge) share common needs: they use an LLM to generate output, they might need to retry on errors, and they should track token usage. The BaseComponent<\/code> class (defined in <a href=\"https:\/\/github.com\/iason-solomos\/Deb8flow\/blob\/main\/nodes\/base_component.py\">nodes\/base_component.py<\/a><\/code>) provides these common features so we don\u2019t repeat code.<\/p>\nclass BaseComponent:\n \"\"\"\n A foundational class for managing LLM-based workflows with token tracking.\n Can handle both Azure OpenAI (AzureChatOpenAI) and OpenAI (ChatOpenAI).\n \"\"\"\n\n def __init__(\n self,\n llm_config: Optional[LLMConfig] = None,\n temperature: float = 0.0,\n max_retries: int = 5,\n ):\n \"\"\"\n Initializes the BaseComponent with optional LLM configuration and temperature.\n\n Args:\n llm_config (Optional[LLMConfig]): Configuration for either Azure or OpenAI.\n temperature (float): Controls the randomness of LLM outputs. Defaults to 0.0.\n max_retries (int): How many times to retry on 429 errors.\n \"\"\"\n logger = logging.getLogger(self.__class__.__name__)\n tracer = trace.get_tracer(__name__, tracer_provider=get_tracer_provider())\n\n self.logger = logger\n self.tracer = tracer\n self.llm: Optional[ChatOpenAI] = None\n self.output_parser: Optional[StrOutputParser] = None\n self.state: Optional[DebateState] = None\n self.prompt_template: Optional[ChatPromptTemplate] = None\n self.chain: Optional[RunnableSequence] = None\n self.documents: Optional[List] = None\n self.prompt_tokens = 0\n self.completion_tokens = 0\n self.max_retries = max_retries\n\n if llm_config is not None:\n self.llm = self._init_llm(llm_config, temperature)\n self.output_parser = StrOutputParser()\n\n def _init_llm(self, config: LLMConfig, temperature: float):\n \"\"\"\n Initializes an LLM instance for either Azure OpenAI or standard OpenAI.\n \"\"\"\n if isinstance(config, AzureOpenAILLMConfig):\n # If it's Azure, use the AzureChatOpenAI class\n return AzureChatOpenAI(\n deployment_name=config.deployment_name,\n azure_endpoint=config.azure_endpoint,\n openai_api_version=config.openai_api_version,\n openai_api_key=config.openai_api_key,\n temperature=temperature,\n )\n elif isinstance(config, OpenAILLMConfig):\n # If it's standard OpenAI, use the ChatOpenAI class\n return ChatOpenAI(\n model_name=config.model_name,\n openai_api_key=config.openai_api_key,\n temperature=temperature,\n )\n else:\n raise ValueError(\"Unsupported LLMConfig type.\")\n\n def validate_initialization(self) -> None:\n \"\"\"\n Ensures we have an LLM and an output parser.\n \"\"\"\n if not self.llm:\n raise ValueError(\"LLM is not initialized. Ensure `llm_config` is provided.\")\n if not self.output_parser:\n raise ValueError(\"Output parser is not initialized.\")\n\n def execute_chain(self, inputs: Any) -> Any:\n \"\"\"\n Executes the LLM chain, tracks token usage, and retries on 429 errors.\n \"\"\"\n if not self.chain:\n raise ValueError(\"No chain is initialized for execution.\")\n\n retry_wait = 1 # Initial wait time in seconds\n\n for attempt in range(self.max_retries):\n try:\n with get_openai_callback() as cb:\n result = self.chain.invoke(inputs)\n self.logger.info(\"Prompt Token usage: %s\", cb.prompt_tokens)\n self.logger.info(\"Completion Token usage: %s\", cb.completion_tokens)\n self.prompt_tokens = cb.prompt_tokens\n self.completion_tokens = cb.completion_tokens\n\n return result\n\n except Exception as e:\n # If the error mentions 429, do exponential backoff and retry\n if \"429\" in str(e):\n self.logger.warning(\n f\"Rate limit reached. Retrying in {retry_wait} seconds... \"\n f\"(Attempt {attempt + 1}\/{self.max_retries})\"\n )\n time.sleep(retry_wait)\n retry_wait *= 2\n else:\n self.logger.error(f\"Unexpected error: {str(e)}\")\n raise e\n\n raise Exception(\"API request failed after maximum number of retries\")\n\n def create_chain(\n self, system_template: str, human_template: str\n ) -> RunnableSequence:\n \"\"\"\n Creates a chain for unstructured outputs.\n \"\"\"\n self.validate_initialization()\n self.prompt_template = ChatPromptTemplate.from_messages(\n [\n (\"system\", system_template),\n (\"human\", human_template),\n ]\n )\n self.chain = self.prompt_template | self.llm | self.output_parser\n return self.chain\n\n def create_structured_output_chain(\n self, system_template: str, human_template: str, output_model: Type[BaseModel]\n ) -> RunnableSequence:\n \"\"\"\n Creates a chain that yields structured outputs (parsed into a Pydantic model).\n \"\"\"\n self.validate_initialization()\n self.prompt_template = ChatPromptTemplate.from_messages(\n [\n (\"system\", system_template),\n (\"human\", human_template),\n ]\n )\n self.chain = self.prompt_template | self.llm.with_structured_output(output_model)\n return self.chain\n\n def build_return_with_tokens(self, node_specific_data: dict) -> dict:\n \"\"\"\n Convenience method to add token usage info into the return values.\n \"\"\"\n return {\n **node_specific_data,\n \"prompt_tokens\": self.prompt_tokens,\n \"completion_tokens\": self.completion_tokens,\n }\n\n def __call__(self, state: DebateState) -> None:\n \"\"\"\n Updates the node's local copy of the state.\n \"\"\"\n self.state = state\n for key, value in state.items():\n setattr(self, key, value)\n<\/code><\/pre>\nKey features of BaseComponent<\/code>:<\/p>\n\n- It stores an LLM client (e.g. an OpenAI
ChatOpenAI<\/code> instance) initialized with a given model and API key, as well as an output parser.<\/li>\n- It provides a method
create_chain(system_template, human_template)<\/code> which sets up a LangChain prompt chain<\/strong> (a RunnableSequence<\/code>) combining a system prompt and a human prompt. This chain is what actually generates outputs when run.<\/li>\n- It has an
execute_chain(inputs)<\/code> method that invokes the chain and includes logic to retry<\/strong> if the OpenAI API returns a rate-limit error (HTTP 429). This is done with exponential backoff up to a max_retries<\/code> count.<\/li>\n- It keeps track of token usage (prompt tokens and completion tokens) for logging or analysis.<\/li>\n
- The
__call__<\/code> method of BaseComponent (which each subclass will call via super().__call__(state)<\/code>) can perform any setup needed before the node\u2019s main logic runs (like ensuring the LLM is initialized).<\/li>\n<\/ul>\n
I\u2019ve In Deb8flow, two agents\u2014Pro and Con\u2014square off on a given topic while a Moderator manages turn-taking. A dedicated Fact Checker reviews every claim in real time using GPT-4o\u2019s new browsing capabilities, and a final Judge evaluates the arguments for quality and coherence. If an agent repeatedly makes factual errors, they\u2019re automatically disqualified\u2014ensuring the debate stays grounded in truth.<\/p>\n This article offers an in-depth look at the advanced architecture and dynamic workflows that power autonomous AI debates. I\u2019ll walk you through how Deb8flow\u2019s modular design leverages LangGraph\u2019s state management and conditional routing, alongside GPT-4o\u2019s capabilities.<\/p>\n Even if you\u2019re new to AI agents or LangGraph (see resources [1] and [2] for primers), I\u2019ll explain the key concepts clearly. And if you\u2019d like to explore further, the full project is available on GitHub: iason-solomos\/Deb8flow<\/strong><\/a>.<\/p>\n Ready to see how AI agents can debate autonomously in practice?<\/p>\n Let\u2019s dive in.<\/strong><\/p>\n In Deb8flow, we orchestrate a formal debate<\/strong> between two AI agents \u2013 one arguing Pro<\/strong> and one Con<\/strong> \u2013 complete with a Moderator<\/strong>, a Fact Checker<\/strong>, and a final Judge<\/strong>. The debate unfolds autonomously, with each agent playing a role in a structured format.<\/p>\n At its core, Deb8flow is a LangGraph-powered agent system, built atop LangChain, using GPT-4o to power each role\u2014Pro, Con, Judge, and beyond. We use GPT-4o\u2019s preview model with browsing capabilities to enable real-time fact-checking. In essence, the Pro and Con agents debate; after each statement, a fact-checker agent uses GPT-4o\u2019s web search to catch any hallucinations or inaccuracies in that statement in real time<\/strong>.\u200b The debate only continues once the statement is verified. The whole process is coordinated by a LangGraph-defined workflow that ensures proper turn-taking and conditional logic.<\/p>\n \n The debate workflow goes through these stages:<\/p>\n After every single speech<\/strong>, the Fact Checker<\/strong> agent steps in to verify the factual accuracy of that statement\u200b. If a debater\u2019s claim doesn\u2019t hold up (e.g. cites a wrong statistic or \u201challucinates\u201d a fact), the workflow triggers a retry<\/em>: the speaker has to correct or modify their statement. (If either debater accumulates 3 fact-check failures, they are automatically disqualified<\/strong> for repeatedly spreading inaccuracies, and their opponent wins by default.) This mechanism keeps our AI debaters honest and grounded in reality!<\/p>\n Before diving into the code, make sure you have the following in place:<\/p>\n Once dependencies are installed and the environment variable is set, you should be ready to run the app. The project structure is organized for clarity:<\/p>\n Deb8flow\/ A quick tour of this structure:<\/p>\n To coordinate a complex multi-turn debate, we need a shared state and a well-defined flow. We\u2019ll start by looking at how Deb8flow defines the debate state<\/strong> and constants, and then see how the LangGraph workflow<\/strong> is constructed.<\/p>\n Deb8flow uses a shared state<\/strong> (https:\/\/langchain-ai.github.io\/langgraph\/concepts\/low_level\/#state <\/a>) in the form of a Python Link: debate_state.py<\/strong><\/a><\/p>\n Key fields that we need to have in the By structuring the debate\u2019s state, agents find it easy to access the conversation history or check the current stage, and the control logic can update the state between turns. The state is essentially the memory of the debate.<\/p>\n To avoid \u201cmagic strings\u201d scattered in the code, we define some constants in debate_constants.py<\/strong><\/a>:<\/strong><\/p>\n We also set up LLM configuration in llm_config.py<\/a>. Here, we define classes for OpenAI or Azure OpenAI configs and then create a dictionary With state and configs in place, we construct the debate workflow graph<\/a>. LangGraph\u2019s StateGraph<\/strong> is the backbone that connects all our agent nodes in the order they should execute. Here\u2019s how we set it up:<\/p>\n Let\u2019s break down what\u2019s happening:<\/p>\n When the workflow runs, control will pass along these edges in order, but whenever we hit a router or moderator node<\/strong>, that node will output a command telling the graph which node to go to next (overriding the default edge). This is how we create conditional loops: the In summary, at a high level we\u2019ve defined an agentic workflow<\/strong>: a graph of autonomous agents where control can branch and loop based on the agents\u2019 outputs. Now, let\u2019s explore what each of these agent nodes actually does.<\/p>\n Each stage or role in the debate is encapsulated in a node (agent). In LangGraph, nodes are often simple functions, but I wanted a more object-oriented approach for clarity and reusability. So in Deb8flow, every node is a class<\/strong> with a Let\u2019s go through the key agents one by one.<\/p>\n Most of our agent nodes (like the debaters and judge) share common needs: they use an LLM to generate output, they might need to retry on errors, and they should track token usage. The Key features of High-Level Overview: Autonomous Debates with Multiple Agents<\/h2>\n
High-level debate flow graph. Each rectangle is an agent node (Pro\/Con debaters, Fact Checker, Judge, etc.), and diamonds are control nodes (Moderator and a router after fact-checking). Solid arrows denote the normal progression, while dashed arrows indicate retries if a claim fails fact-check. The Judge node outputs the final verdict, then the workflow ends.<\/em>
Image generated by the author with DALL-E<\/figcaption><\/figure>\n\n
Prerequisites and Setup<\/h2>\n
\n
git clone https:\/\/github.com\/iason-solomos\/Deb8flow.git<\/code>). The repo includes a requirements.txt<\/code> for all required packages. Key dependencies include LangChain\/LangGraph (for building the agent graph) and the OpenAI Python client.<\/li>\npip install -r requirements.txt<\/code> to install the necessary libraries.<\/li>\n.env<\/code> file in the project root to hold your OpenAI API credentials. It should be of the form: OPENAI_API_KEY_GPT4O = \"sk-\u2026\"<\/code>\n<\/li>\n
\u251c\u2500\u2500 configurations\/
\u2502 \u251c\u2500\u2500 debate_constants.py
\u2502 \u2514\u2500\u2500 llm_config.py
\u251c\u2500\u2500 nodes\/
\u2502 \u251c\u2500\u2500 base_component.py
\u2502 \u251c\u2500\u2500 topic_generator_node.py
\u2502 \u251c\u2500\u2500 pro_debater_node.py
\u2502 \u251c\u2500\u2500 con_debater_node.py
\u2502 \u251c\u2500\u2500 debate_moderator_node.py
\u2502 \u251c\u2500\u2500 fact_checker_node.py
\u2502 \u251c\u2500\u2500 fact_check_router_node.py
\u2502 \u2514\u2500\u2500 judge_node.py
\u251c\u2500\u2500 prompts\/
\u2502 \u251c\u2500\u2500 topic_generator_prompts.py
\u2502 \u251c\u2500\u2500 pro_debater_prompts.py
\u2502 \u251c\u2500\u2500 con_debater_prompts.py
\u2502 \u2514\u2500\u2500 \u2026 (prompts for other agents)
\u251c\u2500\u2500 tests\/ (contains unit and whole workflow tests)
\u2514\u2500\u2500 debate_workflow.py<\/p>\nconfigurations\/<\/code><\/strong> holds constant definitions and LLM configuration classes.<\/p>\nnodes\/<\/code><\/strong> contains the implementation of each agent or functional node in the debate (each of these is a module defining one agent\u2019s behavior).<\/p>\nprompts\/<\/code><\/strong> stores the prompt templates for the language model (so each agent knows how to prompt GPT-4o for its specific task).<\/p>\ndebate_workflow.py<\/code><\/strong> ties everything together by defining the LangGraph workflow (the graph of nodes and transitions).<\/p>\ndebate_state.py<\/code><\/strong> defines the shared data structure that the agents will be using on each run.<\/p>\ntests\/<\/code><\/strong> includes some basic tests and example runs to help you verify everything is working.<\/p>\nUnder the Hood: State Management and Workflow Setup<\/h2>\n
Defining the Debate State Schema (
debate_state.py<\/code>)<\/h3>\nTypedDict<\/code> that all agents can read from and update. This state tracks the debate\u2019s progress and context \u2013 things like the topic, the history of messages, whose turn it is, etc. By centralizing this information, each agent node can make decisions based on the current state of the debate.<\/p>\nfrom typing import TypedDict, List, Dict, Literal\n\n\nDebateStage = Literal[\"opening\", \"rebuttal\", \"counter\", \"final_argument\"]\n\nclass DebateMessage(TypedDict):\n speaker: str # e.g. pro or con\n content: str # The message each speaker produced\n validated: bool # Whether the FactChecker ok\u2019d this message\n stage: DebateStage # The stage of the debate when this message was produced\n\nclass DebateState(TypedDict):\n debate_topic: str\n positions: Dict[str, str]\n messages: List[DebateMessage]\n opening_statement_pro_agent: str\n stage: str # \"opening\", \"rebuttal\", \"counter\", \"final_argument\"\n speaker: str # \"pro\" or \"con\"\n times_pro_fact_checked: int # The number of times the pro agent has been fact-checked. If it reaches 3, the pro agent is disqualified.\n times_con_fact_checked: int # The number of times the con agent has been fact-checked. If it reaches 3, the con agent is disqualified.<\/code><\/pre>\nDebateState<\/code> include:<\/p>\n\n
debate_topic<\/code> (str): The topic being debated.<\/li>\nmessages<\/code> (List[DebateMessage]): A list of all messages exchanged so far. Each message is a dictionary with fields for speaker<\/code> (e.g. \"pro\"<\/code> or \"con\"<\/code> or \"fact_checker\"<\/code>), the message content<\/code> (text), a validated<\/code> flag (whether it passed fact-check), and the stage<\/code> of the debate when it was produced.<\/li>\nstage<\/code> (str): The current debate stage (one of \"opening\"<\/code>, \"rebuttal\"<\/code>, \"counter\"<\/code>, \"final_argument\"<\/code>).<\/li>\nspeaker<\/code> (str): Whose turn it is currently (\"pro\"<\/code> or \"con\"<\/code>).<\/li>\ntimes_pro_fact_checked<\/code> \/ times_con_fact_checked<\/code> (int): Counters for how many times each side has been caught with a false claim. (In our rules, if a debater fails fact-check 3 times, they could be disqualified or automatically lose.)<\/li>\npositions<\/code> (Dict[str, str]): (Optional) A mapping of each side\u2019s general stance (e.g., \"pro\": \"In favor of the topic\"<\/code>).<\/li>\n<\/ul>\nConstants and Configuration<\/h3>\n
debate_constants.py<\/code>. For example, constants for stage names (STAGE_OPENING = \"opening\"<\/code>, etc.), speaker identifiers (SPEAKER_PRO = \"pro\"<\/code>, SPEAKER_CON = \"con\"<\/code>, etc.), and node names (NODE_PRO_DEBATER = \"pro_debater_node\"<\/code>, etc.). These make the code easier to maintain and read.<\/p>\n# Stage names\nSTAGE_OPENING = \"opening\"\nSTAGE_REBUTTAL = \"rebuttal\"\nSTAGE_COUNTER = \"counter\"\nSTAGE_FINAL_ARGUMENT = \"final_argument\"\nSTAGE_END = \"end\"\n\n# Speakers\nSPEAKER_PRO = \"pro\"\nSPEAKER_CON = \"con\"\nSPEAKER_JUDGE = \"judge\"\n\n# Node names\nNODE_PRO_DEBATER = \"pro_debater_node\"\nNODE_CON_DEBATER = \"con_debater_node\"\nNODE_DEBATE_MODERATOR = \"debate_moderator_node\"\nNODE_JUDGE = \"judge_node\"\n<\/code><\/pre>\nllm_config_map<\/code> mapping model names to their config. For instance, we map \"gpt-4o\"<\/code> to an OpenAILLMConfig<\/code> that holds the model name and API key. This way, whenever we need to initialize a GPT-4o agent, we can just do llm_config_map[\"gpt-4o\"]<\/code> to get the right config. All our main agents (debaters, topic generator, judge) use this same GPT-4o configuration.<\/p>\nimport os\nfrom dataclasses import dataclass\nfrom typing import Union\n\n@dataclass\nclass OpenAILLMConfig:\n \"\"\"\n A data class to store configuration details for OpenAI models.\n\n Attributes:\n model_name (str): The name of the OpenAI model to use.\n openai_api_key (str): The API key for authenticating with the OpenAI service.\n \"\"\"\n model_name: str\n openai_api_key: str\n\n\nllm_config_map = {\n \"gpt-4o\": OpenAILLMConfig(\n model_name=\"gpt-4o\",\n openai_api_key=os.getenv(\"OPENAI_API_KEY_GPT4O\"),\n )\n}\n<\/code><\/pre>\nBuilding the LangGraph Workflow (
debate_workflow.py<\/code>)<\/h3>\nclass DebateWorkflow:\n\n def _initialize_workflow(self) -> StateGraph:\n workflow = StateGraph(DebateState)\n # Nodes\n workflow.add_node(\"generate_topic_node\", GenerateTopicNode(llm_config_map[\"gpt-4o\"]))\n workflow.add_node(\"pro_debater_node\", ProDebaterNode(llm_config_map[\"gpt-4o\"]))\n workflow.add_node(\"con_debater_node\", ConDebaterNode(llm_config_map[\"gpt-4o\"]))\n workflow.add_node(\"fact_check_node\", FactCheckNode())\n workflow.add_node(\"fact_check_router_node\", FactCheckRouterNode())\n workflow.add_node(\"debate_moderator_node\", DebateModeratorNode())\n workflow.add_node(\"judge_node\", JudgeNode(llm_config_map[\"gpt-4o\"]))\n\n # Entry point\n workflow.set_entry_point(\"generate_topic_node\")\n\n # Flow\n workflow.add_edge(\"generate_topic_node\", \"pro_debater_node\")\n workflow.add_edge(\"pro_debater_node\", \"fact_check_node\")\n workflow.add_edge(\"con_debater_node\", \"fact_check_node\")\n workflow.add_edge(\"fact_check_node\", \"fact_check_router_node\")\n workflow.add_edge(\"judge_node\", END)\n return workflow\n\n\n\n async def run(self):\n workflow = self._initialize_workflow()\n graph = workflow.compile()\n # graph.get_graph().draw_mermaid_png(output_file_path=\"workflow_graph.png\")\n initial_state = {\n \"topic\": \"\",\n \"positions\": {}\n }\n final_state = await graph.ainvoke(initial_state, config={\"recursion_limit\": 50})\n return final_state\n<\/code><\/pre>\n\n
StateGraph<\/code> with our DebateState<\/code> type as the state schema.<\/li>\n\"pro_debater_node\"<\/code> is added as ProDebaterNode(llm_config_map[\"gpt-4o\"])<\/code>, meaning the Pro debater agent will use GPT-4o as its underlying model.<\/li>\n\"generate_topic_node\"<\/code>. This means the first step of the workflow is to generate a debate topic.<\/li>\nEND<\/code> marker to terminate the graph.<\/li>\n<\/ul>\nfact_check_router_node<\/code> might send us back to a debater node for a retry, instead of following a straight line. LangGraph supports this by allowing nodes to return a special Command<\/code> object with goto<\/code> instructions.<\/p>\nAgent Nodes Breakdown<\/h2>\n
__call__<\/code> method. All the main agent classes inherit from a common BaseComponent<\/code> for shared functionality. This design makes the system modular: we can easily swap out or extend agents by modifying their class definitions, and each agent class is responsible for its piece of the workflow.<\/p>\n\n
BaseComponent<\/code> \u2013 A Reusable Agent Base Class<\/h3>\nBaseComponent<\/code> class (defined in <a href=\"https:\/\/github.com\/iason-solomos\/Deb8flow\/blob\/main\/nodes\/base_component.py\">nodes\/base_component.py<\/a><\/code>) provides these common features so we don\u2019t repeat code.<\/p>\nclass BaseComponent:\n \"\"\"\n A foundational class for managing LLM-based workflows with token tracking.\n Can handle both Azure OpenAI (AzureChatOpenAI) and OpenAI (ChatOpenAI).\n \"\"\"\n\n def __init__(\n self,\n llm_config: Optional[LLMConfig] = None,\n temperature: float = 0.0,\n max_retries: int = 5,\n ):\n \"\"\"\n Initializes the BaseComponent with optional LLM configuration and temperature.\n\n Args:\n llm_config (Optional[LLMConfig]): Configuration for either Azure or OpenAI.\n temperature (float): Controls the randomness of LLM outputs. Defaults to 0.0.\n max_retries (int): How many times to retry on 429 errors.\n \"\"\"\n logger = logging.getLogger(self.__class__.__name__)\n tracer = trace.get_tracer(__name__, tracer_provider=get_tracer_provider())\n\n self.logger = logger\n self.tracer = tracer\n self.llm: Optional[ChatOpenAI] = None\n self.output_parser: Optional[StrOutputParser] = None\n self.state: Optional[DebateState] = None\n self.prompt_template: Optional[ChatPromptTemplate] = None\n self.chain: Optional[RunnableSequence] = None\n self.documents: Optional[List] = None\n self.prompt_tokens = 0\n self.completion_tokens = 0\n self.max_retries = max_retries\n\n if llm_config is not None:\n self.llm = self._init_llm(llm_config, temperature)\n self.output_parser = StrOutputParser()\n\n def _init_llm(self, config: LLMConfig, temperature: float):\n \"\"\"\n Initializes an LLM instance for either Azure OpenAI or standard OpenAI.\n \"\"\"\n if isinstance(config, AzureOpenAILLMConfig):\n # If it's Azure, use the AzureChatOpenAI class\n return AzureChatOpenAI(\n deployment_name=config.deployment_name,\n azure_endpoint=config.azure_endpoint,\n openai_api_version=config.openai_api_version,\n openai_api_key=config.openai_api_key,\n temperature=temperature,\n )\n elif isinstance(config, OpenAILLMConfig):\n # If it's standard OpenAI, use the ChatOpenAI class\n return ChatOpenAI(\n model_name=config.model_name,\n openai_api_key=config.openai_api_key,\n temperature=temperature,\n )\n else:\n raise ValueError(\"Unsupported LLMConfig type.\")\n\n def validate_initialization(self) -> None:\n \"\"\"\n Ensures we have an LLM and an output parser.\n \"\"\"\n if not self.llm:\n raise ValueError(\"LLM is not initialized. Ensure `llm_config` is provided.\")\n if not self.output_parser:\n raise ValueError(\"Output parser is not initialized.\")\n\n def execute_chain(self, inputs: Any) -> Any:\n \"\"\"\n Executes the LLM chain, tracks token usage, and retries on 429 errors.\n \"\"\"\n if not self.chain:\n raise ValueError(\"No chain is initialized for execution.\")\n\n retry_wait = 1 # Initial wait time in seconds\n\n for attempt in range(self.max_retries):\n try:\n with get_openai_callback() as cb:\n result = self.chain.invoke(inputs)\n self.logger.info(\"Prompt Token usage: %s\", cb.prompt_tokens)\n self.logger.info(\"Completion Token usage: %s\", cb.completion_tokens)\n self.prompt_tokens = cb.prompt_tokens\n self.completion_tokens = cb.completion_tokens\n\n return result\n\n except Exception as e:\n # If the error mentions 429, do exponential backoff and retry\n if \"429\" in str(e):\n self.logger.warning(\n f\"Rate limit reached. Retrying in {retry_wait} seconds... \"\n f\"(Attempt {attempt + 1}\/{self.max_retries})\"\n )\n time.sleep(retry_wait)\n retry_wait *= 2\n else:\n self.logger.error(f\"Unexpected error: {str(e)}\")\n raise e\n\n raise Exception(\"API request failed after maximum number of retries\")\n\n def create_chain(\n self, system_template: str, human_template: str\n ) -> RunnableSequence:\n \"\"\"\n Creates a chain for unstructured outputs.\n \"\"\"\n self.validate_initialization()\n self.prompt_template = ChatPromptTemplate.from_messages(\n [\n (\"system\", system_template),\n (\"human\", human_template),\n ]\n )\n self.chain = self.prompt_template | self.llm | self.output_parser\n return self.chain\n\n def create_structured_output_chain(\n self, system_template: str, human_template: str, output_model: Type[BaseModel]\n ) -> RunnableSequence:\n \"\"\"\n Creates a chain that yields structured outputs (parsed into a Pydantic model).\n \"\"\"\n self.validate_initialization()\n self.prompt_template = ChatPromptTemplate.from_messages(\n [\n (\"system\", system_template),\n (\"human\", human_template),\n ]\n )\n self.chain = self.prompt_template | self.llm.with_structured_output(output_model)\n return self.chain\n\n def build_return_with_tokens(self, node_specific_data: dict) -> dict:\n \"\"\"\n Convenience method to add token usage info into the return values.\n \"\"\"\n return {\n **node_specific_data,\n \"prompt_tokens\": self.prompt_tokens,\n \"completion_tokens\": self.completion_tokens,\n }\n\n def __call__(self, state: DebateState) -> None:\n \"\"\"\n Updates the node's local copy of the state.\n \"\"\"\n self.state = state\n for key, value in state.items():\n setattr(self, key, value)\n<\/code><\/pre>\nBaseComponent<\/code>:<\/p>\n\n
ChatOpenAI<\/code> instance) initialized with a given model and API key, as well as an output parser.<\/li>\ncreate_chain(system_template, human_template)<\/code> which sets up a LangChain prompt chain<\/strong> (a RunnableSequence<\/code>) combining a system prompt and a human prompt. This chain is what actually generates outputs when run.<\/li>\nexecute_chain(inputs)<\/code> method that invokes the chain and includes logic to retry<\/strong> if the OpenAI API returns a rate-limit error (HTTP 429). This is done with exponential backoff up to a max_retries<\/code> count.<\/li>\n__call__<\/code> method of BaseComponent (which each subclass will call via super().__call__(state)<\/code>) can perform any setup needed before the node\u2019s main logic runs (like ensuring the LLM is initialized).<\/li>\n<\/ul>\n