Fifth in a series on how software architecture shapes AI-driven code degradation. The previous post built a score for the cost of a change. This post spends it on the change that matters most.
Most benchmarks measure adding. Give the agent a fresh task. See if it works. Move on.
But adding is the easy part. Real software does not just grow. It changes. A rule you shipped last month gets revised this month. A requirement you thought was settled turns out to be wrong. The bill for software is not paid when you write a feature. It is paid every time you have to change one.
So we measured both. And changing, it turns out, is where the two architectures pull furthest apart.
Two kinds of change
Our experiment feeds each codebase a stream of about sixty rule changes. They come in two kinds.
Most are additive. Add a new rule. A phone format check. A duplicate owner warning. Something that did not exist before.
Some are mutative. Take a rule that already exists and revise it. Change how a telephone number is normalised. Redefine what counts as a duplicate. The rule was there. Now it is different.
That second kind is the real test, and it is the cleanest comparison in the whole experiment. Here is why. When the change is mandated, both arms must make the exact same change. The requirement is identical. Neither arm gets to choose an easier path. So any difference in cost is not about the task. It is purely about the architecture. Same required change. Different bill.
It would be tempting to set these aside. The change was forced, you might say, not the architecture's fault. But that misses the point entirely. The force is equal on both arms. What differs is what each one has to disturb to comply.
The bill
Here is what a typical change costs, scored by the change impact metric from the last post. Lower is better.
| Add a new rule | Change an existing rule | |
|---|---|---|
| OfficeFloor (additive) | 238 | 2,222 |
| Spring (mutative) | 3,214 | 11,644 |
Typical change, measured as the median change impact score across the run.
Read it in two directions.
Down each column, one number stands out. Changing a rule costs far more than adding one. For OfficeFloor, about nine times more. For Spring, about four times more. Changing is the expensive part for both architectures. That confirms the whole premise. The cost of software is in the changes, not the additions.
Across each row, the architecture gap is stark. Spring pays multiples more than OfficeFloor whether it is adding or changing. And one comparison is worth pausing on. It costs OfficeFloor less to change an existing rule than it costs Spring to add a brand new one. The additive architecture's hardest task is cheaper than the mutative architecture's easiest one.
An honest wrinkle
The gap does not widen on changes. It narrows. On additions, Spring pays about thirteen times more than OfficeFloor. On changes, about five times more.
That is not a problem for the thesis. It is the thesis being fair. When a rule must change, OfficeFloor cannot dodge the work either. It has to open up the wired function that owns that rule and edit it. So it loses some of its advantage. Both arms are mutating now.
But look at the absolute numbers, not the ratios. The largest gap in the whole table is on changes. OfficeFloor changing costs about 2,200. Spring changing costs about 11,600. That distance, roughly nine thousand, is bigger than any gap on the additions. Changes are where both the cost and the divergence are highest.
Why the tangle costs
The reason is the same reason the whole series has been circling. Location.
In the additive code base, a rule lives in its own unit. When the rule changes, you change that one unit. The surrounding context is small, so the change impact score stays low.
In the mutative code base, the rule was folded into a growing handler alongside twenty others. All the other rules tangled in beside it, is exactly what the change impact score weights by. You are not just changing a rule. You are disturbing everything it was mixed with.
This is an old idea with a formal name. Architecture researchers measure how far a change can ripple through a system, with metrics like propagation cost and decoupling level. A well decoupled system localises change. A tangled one spreads it. What is new here is watching an AI agent pay that ripple, change by change, in a controlled comparison where the only difference is the architecture.
And it compounds
There is a second effect hiding in the numbers, and it is the worst part for the mutative arm. The handler keeps growing. Every rule that lands in it adds to the surrounding context. So the next revision is weighted by an even heavier tangle. The cost of changing a rule goes up over time, simply because more rules have piled in beside it.
The additive arm has no such spiral. Each rule sits in its own unit, no matter how many other rules exist. Changing one is not made more expensive by the presence of the others. The cost of change stays flat as the system grows. That is the property you actually want when you do not know which requirements will change next. And with an AI agent doing the changing, you rarely do.
The takeaway
Adding features is easy. Any architecture can bolt on new code, and an AI agent will happily do it. The test of an architecture is what happens when the requirements you already built have to change. That is not an edge case. That is the ordinary life of software.
On that test, in this experiment, the additive architecture pays a fraction of the cost, and it does not get more expensive as the system grows. That is the practical shape of the whole result. The full picture is in the hub post.
No comments:
Post a Comment