The anchor post for a series on how software architecture shapes AI-driven code degradation. This is a direct sequel to the first run.
The first run ended with two loose ends.
The headline metric, erosion, washed out. It could not separate the two architectures. And correctness did not diverge either. Both arms kept passing their tests. We called the structural damage a latent risk. Real, but not yet biting.
So we ran it again. Three times longer. And this time the signal is not latent. It is loud. But not everywhere, and not in the way the first run expected.
The experiment, in one paragraph
Hold the coding agent fixed. Use the same model for every change. Make architecture the only thing that varies. One arm is Spring, where new rules land as methods on a controller. The other is OfficeFloor, where new rules attach as separate wired functions. Both implement the same feature, one create-owner endpoint, under the same stream of about sixty accumulating rule changes. Ten independent chains per arm. Sixty changes each. Two arms. Twelve hundred agent sessions in total. Then measure how each code base degrades.
Result 1: the structural signal is decisive
First, the statistics, done properly. For each metric we fit a degradation slope to each arm. Then we bootstrap the difference between the arms. If that difference interval excludes zero, the two architectures really do degrade at different rates. This is the actual between arm test.
| Metric | Spring minus OfficeFloor slope | 95% CI | Reading |
|---|---|---|---|
| Endpoint handler complexity | 0.122 | [0.070, 0.181] | Spring's front door bloats. OfficeFloor's stays flat. |
| God class weight (WMC) | 1.07 | [0.83, 1.28] | Spring grows a heavy class. OfficeFloor spreads out. |
| Handler scoped erosion | 0.0033 | [0.0016, 0.0049] | Spring's handler class erodes. OfficeFloor's does not move. |
| Change impact score | 359 | [234, 507] | Each rule costs Spring far more structural disturbance. |
Every one of these excludes zero. The concentration story holds. As rules pile up, Spring pushes complexity into one method and one class. OfficeFloor keeps adding small separate units. That is the thesis, and this time it is measured cleanly, with the right test.
Result 2: erosion was the wrong ruler
The first run's headline metric did not just wash out this time. It gave the backwards answer. Whole application erosion said OfficeFloor was degrading faster. It cleared every statistical bar while pointing the wrong way.
That is not a footnote. It is a lesson in metric validity, and it has its own post. The short version is that erosion is blind to where complexity sits, and it gets swamped by branchy little algorithms that both arms share. We take it apart in Why Erosion Washed Out: The Leaf-Function Trap.
Result 3: a metric that measures the right thing
If erosion cannot see where complexity lands, we need something that can. So we built a change impact score. It does not ask how complex the new code is. It asks how much existing, entangled code the change had to disturb, weighted by how tangled that code already was. A new isolated unit costs almost nothing. Surgery on a god method costs a lot.
That is the metric behind the change impact row above. We describe how it is built, and why it resists being fooled by shallow file splitting, in a dedicated post.
It is not just a number we invented and liked. We checked whether it predicts real pain, using signals it does not have access to. It does. A change the metric scores as high impact independently costs the agent more money, takes more model time, forces more re-reading of the code, and is far more likely to break a rule it was never asked to touch. Changes that actually broke an untouched rule scored roughly seven to ten times higher on impact than changes that did not.
Result 4: what did not diverge
Now the honest part. Not everything separated. Several things did not, and we will not pretend otherwise.
The agent did not spend significantly more per change on one arm than the other over time. The comprehension cost did not diverge. The number of packages a change reached into did not diverge. Temporal coupling did not diverge. Their between arm slope differences all include zero. So we make no claim that OfficeFloor is cheaper to grow or less coupled over time. The data does not support it.
There is one more nuance worth stating plainly. OfficeFloor disturbs less existing code per change in absolute terms. But that number grows faster for OfficeFloor than for Spring. Its blast radius is smaller, yet not perfectly flat. Additive is not free.
What it means
Put the significant and the not-significant together and a clear picture forms.
Architecture decisively governs how change accumulates. In the mutative arm, complexity concentrates, the god method grows, and each new change disturbs more entangled code. In the additive arm, it does not. That is real, measured, and statistically clean.
Architecture does not yet govern whether the code breaks. Correctness fell in both arms as the horizon grew, and it fell together. The gap in genuinely unintended breakage was small. The cost of the mutative arm is structural, not yet functional.
This matches what others are finding. Recent work reports that functional correctness is decoupled from structural quality, and that sheer code volume predicts architectural decay. We do not claim those findings as ours. We confirm them, and we add the controlled comparison they were missing. We hold the agent fixed and change only the architecture. That is what lets us pin the divergence on architecture itself.
The series
This post is the map. Each result above has its own deep dive.
- Why Erosion Washed Out: The Leaf-Function Trap. How the standard metric gave the backwards answer, and the fix.
- Complexity Is Not the Metric. Cohesion Is. Why complexity is the wrong target, and what to measure instead.
- Measuring the Blast Radius of Change. The change impact metric, and how we validated it.
- Adding Features Is Easy. Changing Them Is the Test. Why changing requirements is where the additive style pays off most.
A note on reproducibility. All results come from a single run, identified as blind-202608100006. Every structural number is recomputed from the committed code and its history, so any new metric can be applied to the same past run without re-invoking the agent. The correctness numbers come from a black box acceptance suite the agent never sees.
Update, 23 August 2026. Three of the numbers above are scoped to the endpoint's entry handler: handler complexity, handler class weight, and handler-scoped erosion. A reader pointed out the flaw in that scoping, and they were right. A pipeline architecture can keep its first function pristine by pushing work into the second one, so a front-door measurement flatters it.
We measured the whole path, following every method call from every node. Two things came back. OfficeFloor's entry node is complexity 1.3 while its worst pipeline node reaches about 13, and once helpers are included its create path carries the same total complexity as Spring's controller, growing at the same rate per rule (3.03 versus 3.25, difference including zero). The additive architecture is not simpler, and in the blind run its worst single method is slightly worse.
What survives, and is a stronger result than the one it replaces: the same complexity is packaged into one unit or twenty-two. After sixty rules, changing one rule in Spring means facing about 201 points of complexity; in OfficeFloor, about 7. That gap replicates in a second run under a different test protocol and cannot be produced by relocating work downstream, because the measure follows the work wherever it goes.
The change-impact and blast-radius results above are unaffected: they are computed over the whole diff against whole-application context, not over a scoped path.
Full working in The Same Complexity. One Unit or Twenty-Two.
No comments:
Post a Comment