Reflective generative AI software components as a development paradigm<\/em><\/h4>\nNowhere has the proliferation of generative AI tooling been more aggressive than in the world of software development. It began with GitHub Copilot\u2019s supercharged autocomplete<\/a>, then exploded into direct code-along integrated tools like Aider<\/a> and Cursor<\/a> that allow software engineers to dictate instructions and have the generated changes applied live, in-editor. Now tools like Devin.ai<\/a> aim to build autonomous software generating platforms which can independently consume feature requests or bug tickets and produce ready-to-review code.<\/p>\nThe grand aspiration of these AI tools is, in actuality, no different from the aspirations of all the software that has ever written by humans: to automate human work. When you scheduled that daily CSV parsing script for your employer back in 2005, you were offloading a tiny bit of the labor owned by our species to some combination of silicon and electricity. Where generative AI tools differ is that they aim to automate the work of automation. Setting this goal as our north star enables more abstract thinking about the inherit challenges and possible solutions of generative AI software development.<\/p>\n
\u2b50 Our North Star: Automate the process of automation<\/p>\n
The Doctor-Patient strategy<\/h3>\n
Most contemporary tools approach our automation goal by building stand-alone \u201ccoding bots.\u201d The evolution of these bots represents an increasing success at converting natural language instructions into subject codebase modifications. Under the hood, these bots are platforms with agentic mechanics (mostly search, RAG, and prompt chains). As such, evolution focuses on improving the agentic elements\u200a\u2014\u200arefining RAG chunking, prompt tuning\u00a0etc.<\/p>\n
This strategy establishes the GenAI tool and the subject codebase as two distinct entities, with a unidirectional relationship between them. This relationship is similar to how a doctor operates on a patient, but never the other way around\u200a\u2014\u200ahence the Doctor-Patient strategy.<\/p>\n![\"\"](\"https:\/\/i0.wp.com\/cdn-images-1.medium.com\/max\/1024\/0%2AMP64lJQnUxng8ICe.png?ssl=1\")
The Doctor-Patient strategy of agentic coding approaches code as an external corpus. Image by <\/strong>ethan@pirate.baby<\/strong><\/a><\/figcaption><\/figure>\nA few reasons come to mind that explain why this Doctor-Patient strategy has been the first (and seemingly only) approach towards automating software automation via\u00a0GenAI:<\/p>\n
\n- \nNovel Integration<\/strong>: Software codebases have been around for decades, while using agentic platforms to modify codebases is an extremely recent concept. So it makes sense that the first tools would be designed to act on existing, independent codebases.<\/li>\n
- \nMonetization<\/strong>: The Doctor-Patient strategy has a clear path to revenue. A seller has a GenAI agent platform\/code bot, a buyer has a codebase, the seller\u2019s platform operates on buyers\u2019 codebase for a\u00a0fee.<\/li>\n
- \nSocial Analog<\/strong>: To a non-developer, the relationship in the Doctor-Patient strategy resembles one they already understand between users and Software Developers. A Developer knows how to code, a user asks for a feature, the developer changes the code to make the feature happen. In this strategy, an agent \u201cknows how to code\u201d and can be swapped directly into that mental\u00a0model.<\/li>\n
- \nFalse Extrapolation<\/strong>: At a small enough scale, the Doctor-Patient model can produce impressive results. It is easy to make the incorrect assumption that simply adding resources will allow those same results to scale to an entire codebase.<\/li>\n<\/ul>\n
The independent and unidirectional relationship between agentic platform\/tool and codebase that defines the Doctor-Patient strategy is also the greatest limiting factor of this strategy, and the severity of this limitation has begun to present itself as a dead end. Two years of agentic tool use in the software development space have surfaced antipatterns that are increasingly recognizable as \u201cbot rot\u201d\u200a\u2014\u200aindications of poorly applied and problematic generated code.<\/p>\n![\"\"](\"https:\/\/i0.wp.com\/cdn-images-1.medium.com\/max\/1024\/0%2AeQi44ZQ7UEerpTCZ.png?ssl=1\")
Bot Rot: the degradation of codebase subjected to generative AI alteration. AI generated image created by Midjourney v6.1<\/strong><\/figcaption><\/figure>\nBot rot stems from agentic tools\u2019 inability to account for, and interact with, the macro architectural design of a project. These tools pepper prompts with lines of context from semantically similar code snippets, which are utterly useless in conveying architecture without a high-level abstraction. Just as a chatbot can manifest a sensible paragraph in a new mystery novel but is unable to thread accurate clues as to \u201cwho did it\u201d, isolated code generations pepper the codebase with duplicated business logic and cluttered namespaces. With each generation, bot rot reduces RAG effectiveness and increases the need for human intervention.<\/p>\n
Because bot rotted code requires a greater cognitive load to modify, developers tend to double down on agentic assistance when working with it, and in turn rapidly accelerate additional bot rotting. The codebase balloons, and bot rot becomes obvious: duplicated and often conflicting business logic, colliding, generic and non-descriptive names for modules, objects, and variables, swamps of dead code and boilerplate commentary, a littering of conflicting singleton elements like loggers, settings objects, and configurations. Ironically, sure signs of bot rot are an upward trend in cycle time and an increased need for human direction\/intervention in agentic\u00a0coding.<\/p>\n
A practical example of bot\u00a0rot<\/h3>\n
This example uses Python to illustrate the concept of bot rot, however a similar example could be made in any programming language. Agentic platforms operate on all programming languages in largely the same way and should demonstrate similar\u00a0results.<\/p>\n
In this example, an application processes TPS reports. Currently, the TPS ID value is parsed by several different methods, in different modules, to extract different elements:<\/p>\n
# src\/ingestion\/report_consumer.py
def parse_department_code(self, report_id:str) -> int:
\"\"\"returns the parsed department code from the TPS report id\"\"\"
dep_id = report_id.split(\u201c-\u201d)[-3]
return get_dep_codes()[dep_id]
# src\/reporter\/tps.py
def get_reporting_date(report_id:str) -> datetime.datetime:
\"\"\"converts the encoded date from the tps report id\"\"\"
stamp = int(report_id.split(\u201cts=\u201d)[1].split(\u201c&\u201d)[0])
return datetime.fromtimestamp(stamp)<\/pre>\nA new feature requires parsing the same department code in a different part of the codebase, as well as parsing several new elements from the TPS ID in other locations. A skilled human developer would recognize that TPS ID parsing was becoming cluttered, and abstract all references to the TPS ID into a first-class object:<\/p>\n
# src\/ingestion\/report_consumer.py
from models.tps_report import TPSReport
def parse_department_code(self, report_id:str) -> int:
\"\"\"Deprecated: just access the code on the TPS object in the future\"\"\"
report = TPSReport(report_id)
return report.department_code<\/pre>\nThis abstraction DRYs out the codebase, reducing duplication and shrinking cognitive load. Not surprisingly, what makes code easier for humans to work with also makes it more \u201cGenAI-able\u201d by consolidating the context into an abstracted model. This reduces noise in RAG, improving the quality of resources available for the next generation.<\/p>\n
An agentic tool must complete this same task without architectural insight, or the agency required to implement the above refactor. Given the same task, a code bot will generate additional, duplicated parsing methods or, worse, generate a partial abstraction within one module and not propagate that abstraction. The pattern created is one of a poorer quality codebase, which in turn elicits poorer quality future generations from the tool. Frequency distortion from the repetitive code further damages the effectiveness of RAG. This bot rot spiral will continue until a human hopefully intervenes with a git reset before the codebase devolves into complete\u00a0anarchy.<\/p>\n
An inversion of\u00a0thinking<\/h3>\n
The fundamental flaw in the Doctor-Patient strategy is that it approaches the codebase as a single-layer corpus, serialized documentation from which to generate completions. In reality, software is non-linear and multidimensional\u200a\u2014\u200aless like a research paper and more like our aforementioned mystery novel. No matter how large the context window or effective the embedding model, agentic tools disambiguated from the architectural design of a codebase will always devolve into bot\u00a0rot.<\/p>\n
How can GenAI powered workflows be equipped with the context and agency required to automate the process of automation? The answer stems from ideas found in two well-established concepts in software engineering.<\/p>\n
TDD<\/h3>\n
Test Driven Development is a cornerstone of modern software engineering process. More than just a mandate to \u201cwrite the tests first,\u201d TDD is a mindset manifested into a process. For our purposes, the pillars of TDD look something like\u00a0this:<\/p>\n
\n- A complete codebase consists of application code that performs desired processes, and test code that ensures the application code works as intended.<\/li>\n
- Test code is written to define what \u201cdone\u201d will look like, and application code is then written to satisfy that test\u00a0code.<\/li>\n<\/ul>\n
TDD implicitly requires that application code be written in a way that is highly testable<\/em>. Overly complex, nested business logic must be broken into units that can be directly accessed by test methods. Hooks need to be baked into object signatures, dependencies must be injected, all to facilitate the ability of test code to assure functionality in the application. Herein is the first part of our answer: for agentic processes to be more successful at automating our codebase, we need to write code that is highly GenAI-able<\/strong>.<\/p>\nAnother important element of TDD in this context is that testing must be an implicit part of the software we build. In TDD, there is no option to scratch out a pile of application code with no tests, then apply a third party bot to \u201ctest it.\u201d This is the second part of our answer: Codebase automation must be an element of the software itself, not an external function of a \u2018code\u00a0bot\u2019<\/strong>.<\/p>\nRefactoring<\/h3>\n
The earlier Python TPS report example demonstrates a code refactor, one of the most important higher-level functions in healthy software evolution. Kent Beck describes the process of refactoring as<\/p>\n
\u201cfor each desired change, make the change easy (warning: this may be hard), then make the easy change.\u201d ~ Kent\u00a0Beck<\/em><\/p><\/blockquote>\nThis is how a codebase improves for human needs over time, reducing cognitive load and, as a result, cycle times. Refactoring is also exactly how a codebase is continually optimized for GenAI automation! Refactoring means removing duplication, decoupling and creating semantic \u201cdistance\u201d between domains, and simplifying the logical flow of a program\u200a\u2014\u200aall things that will have a huge positive impact on both RAG and generative processes. The final part of our answer is that codebase architecture (and subsequently, refactoring) must be a first class citizen as part of any codebase automation process<\/strong>.<\/p>\nGenerative Driven Development<\/h3>\n
Given these borrowed\u00a0pillars:<\/p>\n
\n- \nFor agentic processes to be more successful at automating our codebase, we need to write code that is highly GenAI-able<\/strong>.<\/li>\n
- \nCodebase automation must be an element of the software itself, not an external function of a \u2018code\u00a0bot\u2019<\/strong>.<\/li>\n
- \nCodebase architecture (and subsequently, refactoring) must be a first class citizen as part of any codebase automation process<\/strong>.<\/li>\n<\/ul>\n
An alternative strategy to the unidirectional Doctor-Patient takes shape. This strategy, where application code development itself is driven by the goal of generative self-automation, could be called Generative Driven Development, or\u00a0GDD(1)<\/em>.<\/p>\nGDD is an evolution that moves optimization for agentic self-improvement to the center stage, much in the same way as TDD promoted testing in the development process. In fact, TDD becomes a subset of GDD, in that highly GenAI-able code is both highly testable and, as part of GDD evolution, well\u00a0tested.<\/p>\n
To dissect what a GDD workflow could look like, we can start with a closer look at those\u00a0pillars:<\/p>\n
1. Writing code that is highly GenAI-able<\/h3>\n
In a highly GenAI-able codebase, it is easy to build highly effective embeddings and assemble low-noise context, side effects and coupling are rare, and abstraction is clear and consistent. When it comes to understanding a codebase, the needs of a human developer and those of an agentic process have significant overlap. In fact, many elements of highly GenAI-able code will look familiar in practice to a human-focused code refactor. However, the driver behind these principles is to improve the ability of agentic processes to correctly generate code iterations. Some of these principles include:<\/p>\n
\n- \nHigh cardinality in entity naming<\/strong>: Variables, methods, classes must be as unique as possible to minimize RAG context collisions.<\/li>\n
- \nAppropriate semantic correlation in naming<\/strong>: A Dog class will have a greater embedded similarity to the Cat class than a top-level walk function. Naming needs to form intentional, logical semantic relationships and avoid semantic collisions.<\/li>\n
- \nGranular (highly chunkable) documentation<\/strong>: Every callable, method and object in the codebase must ship with comprehensive, accurate heredocs to facilitate intelligent RAG and the best possible completions.<\/li>\n
- \nFull pathing of resources<\/strong>: Code should remove as much guesswork and assumed context as possible. In a Python project, this would mean fully qualified import paths (no relative imports) and avoiding unconventional aliases.<\/li>\n
- \nExtremely predictable architectural patterns<\/strong>: Consistent use of singular\/plural case, past\/present tense, and documented rules for module nesting enable generations based on demonstrated patterns (generating an import of SaleSchema based not on RAG but inferred by the presence of OrderSchema and ReturnSchema)<\/li>\n
- \nDRY code<\/strong>: duplicated business logic balloons both the context and generated token count, and will increase generated mistakes when a higher presence penalty is\u00a0applied.<\/li>\n<\/ul>\n
2. Tooling as an aspect of the\u00a0software<\/h3>\n
Every commercially viable programming language has at least one accompanying test framework; Python has pytest, Ruby has RSpec, Java has JUnit etc. In comparison, many other aspects of the SDLC evolved into stand-alone tools – like feature management done in Jira or Linear, or monitoring via Datadog. Why, then, are testing code part of the codebase, and testing tools part of development dependencies?<\/p>\n
Tests are an integral part of the software circuit, tightly coupled to the application code they cover. Tests require the ability to account for, and interact with, the macro architectural design of a project (sound familiar?) and must evolve in sync with the whole of the codebase.<\/p>\n
For effective GDD, we will need to see similar purpose-built packages that can support an evolved, generative-first development process. At the core will be a system for building and maintaining an intentional meta-catalog of semantic project architecture. This might be something that is parsed and evolved via the AST, or driven by a ULM-like data structure that both humans and code modify over time\u200a\u2014\u200asimilar to a\u00a0.pytest.ini or plugin configs in a pom.xml file in\u00a0TDD.<\/p>\n
This semantic structure will enable our package to run stepped processes that account for macro architecture, in a way that is both bespoke to and evolving with the project itself. Architectural rules for the application such as naming conventions, responsibilities of different classes, modules, services etc. will compile applicable semantics into agentic pipeline executions, and guide generations to meet\u00a0them.<\/p>\n
Similar to the current crop of test frameworks, GDD tooling will abstract boilerplate generative functionality while offering a heavily customizable API for developers (and the agentic processes) to fine-tune. Like your test specs, generative specs could define architectural directives and external context\u200a\u2014\u200alike the sunsetting of a service, or a team pivot to a new design pattern\u200a\u2014\u200aand inform the agentic generations.<\/p>\n
GDD linting will look for patterns that make code less GenAI-able (see Writing code that is highly GenAI-able<\/a>) and correct them when possible, raise them to human attention when\u00a0not.<\/p>\n3. Architecture as a first-class citizen<\/h3>\n
Consider the problem of bot rot through the lens of a TDD iteration. Traditional TDD operates in three steps: red, green, and refactor.<\/p>\n
\n- \nRed<\/strong>: write a test for the new feature that fails (because you haven\u2019t written the feature\u00a0yet)<\/li>\n
- \nGreen<\/strong>: write the feature as quickly as possible to make the test\u00a0pass<\/li>\n
- \nRefactor<\/strong>: align the now-passing code with the project architecture by abstracting, renaming\u00a0etc.<\/li>\n<\/ul>\n
With bot rot only the \u201cgreen\u201d step is present. Unless explicitly instructed, agentic frameworks will not write a failing test first, and without an understanding of the macro architectural design they cannot effectively refactor a codebase to accommodate the generated code. This is why codebases subject to the current crop of agentic tools degrade rather quickly\u200a\u2014\u200athe executed TDD cycles are incomplete. By elevating these missing \u201cbookends\u201d of the TDD cycle in the agentic process and integrating a semantic map of the codebase architecture to make refactoring possible, bot rot will be effectively alleviated. Over time, a GDD codebase will become increasingly easier to traverse for both human and bot, cycle times will decrease, error rates will fall, and the application will become increasingly self-automating.<\/p>\n
A day in the GDD\u00a0life<\/h3>\n
what could GDD development look\u00a0like?<\/em><\/p>\nA GDD Engineer opens their laptop to start the day, cds into our infamous TPS report repo and opens a terminal. Let\u2019s say the Python GDD equivalent of pytest is a (currently fictional) package named\u00a0py-gdd.<\/p>\n
First, they need to pick some work from the backlog. Scanning over the tickets in Jira they decide on \u201cTPS-122: account for underscores in the new TPS ID format.\u201d They start work in the terminal\u00a0with:<\/p>\n
>> git checkout -b feature\/TPS-122\/id-underscores && py-gdd begin TPS-122<\/pre>\nA terminal spinner appears while py-gdd processes. What is py-gdd\u00a0doing?<\/p>\n
\n- Reading the jira ticket\u00a0content<\/li>\n
- Reviewing current semantic architecture to select smart RAG\u00a0context<\/li>\n
- Reviewing the project directives to adjust context and set boundaries<\/li>\n
- Constructing a plan, which is persisted into a gitignored\u00a0.pygdd\u00a0folder<\/li>\n<\/ul>\n
py-gdd responds with a developer-peer level statement about the execution plan, something to the effect\u00a0of:<\/p>\n
\u201cI am going to parameterize all the tests that use TPS IDs with both dashes and underscores, I don\u2019t think we need a stand-alone test for this then. And then I will abstract all the TPS ID parsing to a single TPS\u00a0model.<\/em>\u201d<\/p>\nNotice how this wasn\u2019t an unreadable wall of code + unimportant context + comment\u00a0noise?<\/p>\n
The Engineer scans the plan, which consists of more granular\u00a0steps:<\/p>\n
\n- Updating 12 tests to parameterized dash and underscore TPS\u00a0IDs<\/li>\n
- Ensuring only the new tests\u00a0fail<\/li>\n
- Updating 8 locations in the code where TPS IDs are\u00a0parsed<\/li>\n
- Ensuring all tests\u00a0pass<\/li>\n
- Abstracting the test parameterization to a single test\u00a0fixture<\/li>\n
- Migrating all 12 tests to the new fixture, confirming that they pass with each migration<\/li>\n
- Creating a new TPS\u00a0model<\/li>\n
- Duplicating all the TPS model logic from the application in the new\u00a0model<\/li>\n
- Confirming tests all still\u00a0pass<\/li>\n
- Swapping in the new model at each of the 8 locations, confirming the tests pass with\u00a0each<\/li>\n
- Updating the semantic architecture to be aware of the new TPS model and edges in the graph that associate with\u00a0it<\/li>\n
- Updating method names in 2 of the locations to be more semantically distinct<\/li>\n<\/ol>\n
On confirmation, the terminal spinner appears again. Text appears in\u00a0red:<\/p>\n
Problem in step 3 - I am unable to get all tests to pass after 2 attempts<\/pre>\nThe Engineer looks at the test code and sees that the database test factory is hard-coded to the old ID type and cannot accept overrides\u200a\u2014\u200athis explains why the py-gdd process cannot get the updates to pass. The Engineer explains this via in-terminal chat to py-gdd, and a new sub-plan is\u00a0created:<\/p>\n
\n- Roll back main plain step\u00a01<\/li>\n
- Modify database test factory to accept an override for TPS\u00a0ID<\/li>\n
- Confirm that all tests now\u00a0pass<\/li>\n
- Update main plan step 1 to include overriding database test factory with parameterized value<\/li>\n
- Re-apply main plan step\u00a01<\/li>\n<\/ol>\n
In addition, back in the main plan the step \u201cupdate semantics for database test factory\u201d is added. The Engineer confirms again, and this time the plan completes successfully, and a Pull Request is created in\u00a0GitHub.<\/p>\n
py-gdd follows up with a list of concerns it developed during the plan execution:<\/p>\n
concerns:
- there are several lower-level modules simply named \u201cserver\u201d and this is semantically confusing.
- the tps_loader module aliases pandas as \u201cpand\u201d which is non-standard and may hinder generation.
- there are no test hooks in the highly complex \u201cpersist_schema\u201d method of the InnatechDB class. This is makes both testing and generation difficult.
...<\/pre>\nThe Engineer instructs py-gdd to create tickets for each concern. On to the next\u00a0ticket!<\/p>\n
The CI\/CD of\u00a0GDD<\/h3>\n
In this vision, an Engineer is still very heavily involved in the mechanical processes of GDD. But it is reasonable to assume that as a codebase grows and evolves to become increasingly GenAI-able due to GDD practice, less human interaction will become necessary. In the ultimate expression of Continuous Delivery, GDD could be primarily practiced via a perpetual \u201cGDD server.\u201d Work will be sourced from project management tools like Jira and GitHub Issues, error logs from Datadog and CloudWatch needing investigation, and most importantly generated by the GDD tooling itself. Hundreds of PRs could be opened, reviewed, and merged every day, with experienced human engineers guiding the architectural development of the project over time. In this way, GDD can become a realization of the goal to automate automation.<\/p>\n
\n- yes, this really is a clear form of machine learning, but that term has been so painfully overloaded that I hesitate to associate any new idea with those\u00a0words.<\/em><\/li>\n<\/ol>\n
originally published on <\/em>pirate.baby<\/em><\/a>, my tech and tech-adjacent blog<\/em><\/p>\n![\"\"](\"https:\/\/medium.com\/_\/stat?event=post.clientViewed&referrerSource=full_rss&postId=0c948fb9a735\")
<\/p>\n
\nGDD: Generative Driven Design<\/a> was originally published in Towards Data Science<\/a> on Medium, where people are continuing the conversation by highlighting and responding to this story.<\/p>\n<\/div>\n
The grand aspiration of these AI tools is, in actuality, no different from the aspirations of all the software that has ever written by humans: to automate human work. When you scheduled that daily CSV parsing script for your employer back in 2005, you were offloading a tiny bit of the labor owned by our species to some combination of silicon and electricity. Where generative AI tools differ is that they aim to automate the work of automation. Setting this goal as our north star enables more abstract thinking about the inherit challenges and possible solutions of generative AI software development.<\/p>\n
\u2b50 Our North Star: Automate the process of automation<\/p>\n
The Doctor-Patient strategy<\/h3>\n
Most contemporary tools approach our automation goal by building stand-alone \u201ccoding bots.\u201d The evolution of these bots represents an increasing success at converting natural language instructions into subject codebase modifications. Under the hood, these bots are platforms with agentic mechanics (mostly search, RAG, and prompt chains). As such, evolution focuses on improving the agentic elements\u200a\u2014\u200arefining RAG chunking, prompt tuning\u00a0etc.<\/p>\n
This strategy establishes the GenAI tool and the subject codebase as two distinct entities, with a unidirectional relationship between them. This relationship is similar to how a doctor operates on a patient, but never the other way around\u200a\u2014\u200ahence the Doctor-Patient strategy.<\/p>\n A few reasons come to mind that explain why this Doctor-Patient strategy has been the first (and seemingly only) approach towards automating software automation via\u00a0GenAI:<\/p>\n The independent and unidirectional relationship between agentic platform\/tool and codebase that defines the Doctor-Patient strategy is also the greatest limiting factor of this strategy, and the severity of this limitation has begun to present itself as a dead end. Two years of agentic tool use in the software development space have surfaced antipatterns that are increasingly recognizable as \u201cbot rot\u201d\u200a\u2014\u200aindications of poorly applied and problematic generated code.<\/p>\n Bot rot stems from agentic tools\u2019 inability to account for, and interact with, the macro architectural design of a project. These tools pepper prompts with lines of context from semantically similar code snippets, which are utterly useless in conveying architecture without a high-level abstraction. Just as a chatbot can manifest a sensible paragraph in a new mystery novel but is unable to thread accurate clues as to \u201cwho did it\u201d, isolated code generations pepper the codebase with duplicated business logic and cluttered namespaces. With each generation, bot rot reduces RAG effectiveness and increases the need for human intervention.<\/p>\n Because bot rotted code requires a greater cognitive load to modify, developers tend to double down on agentic assistance when working with it, and in turn rapidly accelerate additional bot rotting. The codebase balloons, and bot rot becomes obvious: duplicated and often conflicting business logic, colliding, generic and non-descriptive names for modules, objects, and variables, swamps of dead code and boilerplate commentary, a littering of conflicting singleton elements like loggers, settings objects, and configurations. Ironically, sure signs of bot rot are an upward trend in cycle time and an increased need for human direction\/intervention in agentic\u00a0coding.<\/p>\n This example uses Python to illustrate the concept of bot rot, however a similar example could be made in any programming language. Agentic platforms operate on all programming languages in largely the same way and should demonstrate similar\u00a0results.<\/p>\n In this example, an application processes TPS reports. Currently, the TPS ID value is parsed by several different methods, in different modules, to extract different elements:<\/p>\n A new feature requires parsing the same department code in a different part of the codebase, as well as parsing several new elements from the TPS ID in other locations. A skilled human developer would recognize that TPS ID parsing was becoming cluttered, and abstract all references to the TPS ID into a first-class object:<\/p>\n This abstraction DRYs out the codebase, reducing duplication and shrinking cognitive load. Not surprisingly, what makes code easier for humans to work with also makes it more \u201cGenAI-able\u201d by consolidating the context into an abstracted model. This reduces noise in RAG, improving the quality of resources available for the next generation.<\/p>\n An agentic tool must complete this same task without architectural insight, or the agency required to implement the above refactor. Given the same task, a code bot will generate additional, duplicated parsing methods or, worse, generate a partial abstraction within one module and not propagate that abstraction. The pattern created is one of a poorer quality codebase, which in turn elicits poorer quality future generations from the tool. Frequency distortion from the repetitive code further damages the effectiveness of RAG. This bot rot spiral will continue until a human hopefully intervenes with a git reset before the codebase devolves into complete\u00a0anarchy.<\/p>\n The fundamental flaw in the Doctor-Patient strategy is that it approaches the codebase as a single-layer corpus, serialized documentation from which to generate completions. In reality, software is non-linear and multidimensional\u200a\u2014\u200aless like a research paper and more like our aforementioned mystery novel. No matter how large the context window or effective the embedding model, agentic tools disambiguated from the architectural design of a codebase will always devolve into bot\u00a0rot.<\/p>\n How can GenAI powered workflows be equipped with the context and agency required to automate the process of automation? The answer stems from ideas found in two well-established concepts in software engineering.<\/p>\n Test Driven Development is a cornerstone of modern software engineering process. More than just a mandate to \u201cwrite the tests first,\u201d TDD is a mindset manifested into a process. For our purposes, the pillars of TDD look something like\u00a0this:<\/p>\n TDD implicitly requires that application code be written in a way that is highly testable<\/em>. Overly complex, nested business logic must be broken into units that can be directly accessed by test methods. Hooks need to be baked into object signatures, dependencies must be injected, all to facilitate the ability of test code to assure functionality in the application. Herein is the first part of our answer: for agentic processes to be more successful at automating our codebase, we need to write code that is highly GenAI-able<\/strong>.<\/p>\n Another important element of TDD in this context is that testing must be an implicit part of the software we build. In TDD, there is no option to scratch out a pile of application code with no tests, then apply a third party bot to \u201ctest it.\u201d This is the second part of our answer: Codebase automation must be an element of the software itself, not an external function of a \u2018code\u00a0bot\u2019<\/strong>.<\/p>\n The earlier Python TPS report example demonstrates a code refactor, one of the most important higher-level functions in healthy software evolution. Kent Beck describes the process of refactoring as<\/p>\n \u201cfor each desired change, make the change easy (warning: this may be hard), then make the easy change.\u201d ~ Kent\u00a0Beck<\/em><\/p><\/blockquote>\n This is how a codebase improves for human needs over time, reducing cognitive load and, as a result, cycle times. Refactoring is also exactly how a codebase is continually optimized for GenAI automation! Refactoring means removing duplication, decoupling and creating semantic \u201cdistance\u201d between domains, and simplifying the logical flow of a program\u200a\u2014\u200aall things that will have a huge positive impact on both RAG and generative processes. The final part of our answer is that codebase architecture (and subsequently, refactoring) must be a first class citizen as part of any codebase automation process<\/strong>.<\/p>\n Given these borrowed\u00a0pillars:<\/p>\n An alternative strategy to the unidirectional Doctor-Patient takes shape. This strategy, where application code development itself is driven by the goal of generative self-automation, could be called Generative Driven Development, or\u00a0GDD(1)<\/em>.<\/p>\n GDD is an evolution that moves optimization for agentic self-improvement to the center stage, much in the same way as TDD promoted testing in the development process. In fact, TDD becomes a subset of GDD, in that highly GenAI-able code is both highly testable and, as part of GDD evolution, well\u00a0tested.<\/p>\n To dissect what a GDD workflow could look like, we can start with a closer look at those\u00a0pillars:<\/p>\n In a highly GenAI-able codebase, it is easy to build highly effective embeddings and assemble low-noise context, side effects and coupling are rare, and abstraction is clear and consistent. When it comes to understanding a codebase, the needs of a human developer and those of an agentic process have significant overlap. In fact, many elements of highly GenAI-able code will look familiar in practice to a human-focused code refactor. However, the driver behind these principles is to improve the ability of agentic processes to correctly generate code iterations. Some of these principles include:<\/p>\n Every commercially viable programming language has at least one accompanying test framework; Python has pytest, Ruby has RSpec, Java has JUnit etc. In comparison, many other aspects of the SDLC evolved into stand-alone tools – like feature management done in Jira or Linear, or monitoring via Datadog. Why, then, are testing code part of the codebase, and testing tools part of development dependencies?<\/p>\n Tests are an integral part of the software circuit, tightly coupled to the application code they cover. Tests require the ability to account for, and interact with, the macro architectural design of a project (sound familiar?) and must evolve in sync with the whole of the codebase.<\/p>\n For effective GDD, we will need to see similar purpose-built packages that can support an evolved, generative-first development process. At the core will be a system for building and maintaining an intentional meta-catalog of semantic project architecture. This might be something that is parsed and evolved via the AST, or driven by a ULM-like data structure that both humans and code modify over time\u200a\u2014\u200asimilar to a\u00a0.pytest.ini or plugin configs in a pom.xml file in\u00a0TDD.<\/p>\n This semantic structure will enable our package to run stepped processes that account for macro architecture, in a way that is both bespoke to and evolving with the project itself. Architectural rules for the application such as naming conventions, responsibilities of different classes, modules, services etc. will compile applicable semantics into agentic pipeline executions, and guide generations to meet\u00a0them.<\/p>\n Similar to the current crop of test frameworks, GDD tooling will abstract boilerplate generative functionality while offering a heavily customizable API for developers (and the agentic processes) to fine-tune. Like your test specs, generative specs could define architectural directives and external context\u200a\u2014\u200alike the sunsetting of a service, or a team pivot to a new design pattern\u200a\u2014\u200aand inform the agentic generations.<\/p>\n GDD linting will look for patterns that make code less GenAI-able (see Writing code that is highly GenAI-able<\/a>) and correct them when possible, raise them to human attention when\u00a0not.<\/p>\n Consider the problem of bot rot through the lens of a TDD iteration. Traditional TDD operates in three steps: red, green, and refactor.<\/p>\n With bot rot only the \u201cgreen\u201d step is present. Unless explicitly instructed, agentic frameworks will not write a failing test first, and without an understanding of the macro architectural design they cannot effectively refactor a codebase to accommodate the generated code. This is why codebases subject to the current crop of agentic tools degrade rather quickly\u200a\u2014\u200athe executed TDD cycles are incomplete. By elevating these missing \u201cbookends\u201d of the TDD cycle in the agentic process and integrating a semantic map of the codebase architecture to make refactoring possible, bot rot will be effectively alleviated. Over time, a GDD codebase will become increasingly easier to traverse for both human and bot, cycle times will decrease, error rates will fall, and the application will become increasingly self-automating.<\/p>\n what could GDD development look\u00a0like?<\/em><\/p>\n A GDD Engineer opens their laptop to start the day, cds into our infamous TPS report repo and opens a terminal. Let\u2019s say the Python GDD equivalent of pytest is a (currently fictional) package named\u00a0py-gdd.<\/p>\n First, they need to pick some work from the backlog. Scanning over the tickets in Jira they decide on \u201cTPS-122: account for underscores in the new TPS ID format.\u201d They start work in the terminal\u00a0with:<\/p>\n A terminal spinner appears while py-gdd processes. What is py-gdd\u00a0doing?<\/p>\n py-gdd responds with a developer-peer level statement about the execution plan, something to the effect\u00a0of:<\/p>\n \u201cI am going to parameterize all the tests that use TPS IDs with both dashes and underscores, I don\u2019t think we need a stand-alone test for this then. And then I will abstract all the TPS ID parsing to a single TPS\u00a0model.<\/em>\u201d<\/p>\n Notice how this wasn\u2019t an unreadable wall of code + unimportant context + comment\u00a0noise?<\/p>\n The Engineer scans the plan, which consists of more granular\u00a0steps:<\/p>\n On confirmation, the terminal spinner appears again. Text appears in\u00a0red:<\/p>\n The Engineer looks at the test code and sees that the database test factory is hard-coded to the old ID type and cannot accept overrides\u200a\u2014\u200athis explains why the py-gdd process cannot get the updates to pass. The Engineer explains this via in-terminal chat to py-gdd, and a new sub-plan is\u00a0created:<\/p>\n In addition, back in the main plan the step \u201cupdate semantics for database test factory\u201d is added. The Engineer confirms again, and this time the plan completes successfully, and a Pull Request is created in\u00a0GitHub.<\/p>\n py-gdd follows up with a list of concerns it developed during the plan execution:<\/p>\n The Engineer instructs py-gdd to create tickets for each concern. On to the next\u00a0ticket!<\/p>\n In this vision, an Engineer is still very heavily involved in the mechanical processes of GDD. But it is reasonable to assume that as a codebase grows and evolves to become increasingly GenAI-able due to GDD practice, less human interaction will become necessary. In the ultimate expression of Continuous Delivery, GDD could be primarily practiced via a perpetual \u201cGDD server.\u201d Work will be sourced from project management tools like Jira and GitHub Issues, error logs from Datadog and CloudWatch needing investigation, and most importantly generated by the GDD tooling itself. Hundreds of PRs could be opened, reviewed, and merged every day, with experienced human engineers guiding the architectural development of the project over time. In this way, GDD can become a realization of the goal to automate automation.<\/p>\n originally published on <\/em>pirate.baby<\/em><\/a>, my tech and tech-adjacent blog<\/em><\/p>\n GDD: Generative Driven Design<\/a> was originally published in Towards Data Science<\/a> on Medium, where people are continuing the conversation by highlighting and responding to this story.<\/p>\n<\/div>\n\n
A practical example of bot\u00a0rot<\/h3>\n
# src\/ingestion\/report_consumer.py
def parse_department_code(self, report_id:str) -> int:
\"\"\"returns the parsed department code from the TPS report id\"\"\"
dep_id = report_id.split(\u201c-\u201d)[-3]
return get_dep_codes()[dep_id]
# src\/reporter\/tps.py
def get_reporting_date(report_id:str) -> datetime.datetime:
\"\"\"converts the encoded date from the tps report id\"\"\"
stamp = int(report_id.split(\u201cts=\u201d)[1].split(\u201c&\u201d)[0])
return datetime.fromtimestamp(stamp)<\/pre>\n# src\/ingestion\/report_consumer.py
from models.tps_report import TPSReport
def parse_department_code(self, report_id:str) -> int:
\"\"\"Deprecated: just access the code on the TPS object in the future\"\"\"
report = TPSReport(report_id)
return report.department_code<\/pre>\nAn inversion of\u00a0thinking<\/h3>\n
TDD<\/h3>\n
\n
Refactoring<\/h3>\n
Generative Driven Development<\/h3>\n
\n
1. Writing code that is highly GenAI-able<\/h3>\n
\n
2. Tooling as an aspect of the\u00a0software<\/h3>\n
3. Architecture as a first-class citizen<\/h3>\n
\n
A day in the GDD\u00a0life<\/h3>\n
>> git checkout -b feature\/TPS-122\/id-underscores && py-gdd begin TPS-122<\/pre>\n
\n
\n
Problem in step 3 - I am unable to get all tests to pass after 2 attempts<\/pre>\n
\n
concerns:
- there are several lower-level modules simply named \u201cserver\u201d and this is semantically confusing.
- the tps_loader module aliases pandas as \u201cpand\u201d which is non-standard and may hinder generation.
- there are no test hooks in the highly complex \u201cpersist_schema\u201d method of the InnatechDB class. This is makes both testing and generation difficult.
...<\/pre>\nThe CI\/CD of\u00a0GDD<\/h3>\n
\n
<\/p>\n
\n