Friday, 7 August 2026

Mutation of existing logic showing to cause erosion

The redesigned PetClinic-Evolve experiment has produced its first complete run of each architecture. It is one chain per arm, so it is a first look and not the final verdict. But it already moves the measure that stayed flat for the whole first experiment, and it moves it in the direction the thesis predicts.

The setup, in one paragraph

Hold the coding agent fixed. Vary the architecture. One agent model builds the same app twice. In the Spring version, request logic lands in controller methods. In the OfficeFloor version, each rule is a small function wired together by YAML. Then sixty change requests land on the same endpoint, create owner, and we watch how each code base ages. Every fourth change is mutative: it revises earlier rules rather than only adding. The idea under test is that Spring concentrates the accumulating logic into one growing method, while OfficeFloor spreads it across many small functions and keeps the entry point flat.

The entry handler: the decisive measure

The cleanest number is the complexity of the one function the create endpoint routes through. In Spring that is the controller method. In OfficeFloor that is the pipeline's create function.

CheckpointSpring create handler (CC)OfficeFloor create function (CC)
100
16112
32132
48219
60279

Cyclomatic complexity counts the independent paths through a function. A value around 27 is a method that is genuinely hard to hold in your head. Spring's create handler climbs to 27 and is still rising at the end. OfficeFloor's create function sat at 2 through the first half and ends at 9. Fitted as a trend, the Spring handler grows about 2.7 times faster per change. This is the mechanism in one line. Rules pile into the Spring handler. They attach beside the OfficeFloor one.

Where the complexity lives

Here is the part that a single summary number hides. Both code bases end with three or four functions above the usual complexity threshold. So a blunt erosion ratio looks similar for the two. But the functions that carry the complexity are not the same kind of thing.

Spring's busiest functions at the end:

ComplexityFunction
27the create controller method itself
19a soundex name-coding routine
12a region-code helper

OfficeFloor's busiest functions at the end:

ComplexityFunction
19a soundex digit routine
13the possible-duplicate rule
13a soundex helper
11the telephone formatting rule

In Spring the single busiest function is the front door itself. In OfficeFloor the busiest functions are isolated, single-purpose units, and the front door stays flat. Notice the soundex routine sits at complexity 19 in both arms. That is the inherent complexity of the algorithm, not erosion, and it shows up in both. The difference is that Spring carries a complexity-27 god method on top of that shared cost. OfficeFloor does not.

Erosion, over the whole run

The erosion measure is the share of complexity that lives in functions above the threshold. It stayed at zero for the entire first experiment. In this run it moves.

CheckpointSpring erosionOfficeFloor erosion
80.000.00
160.240.00
320.280.11
480.290.22
600.330.23

Spring erodes early, from checkpoint 16, and settles around 0.33. OfficeFloor stays at zero until checkpoint 32, then rises to 0.23. So OfficeFloor is not immune. The deep mutations in the back half do push it up. But it erodes later, it erodes lower, and it erodes in a spread out way rather than concentrating in the handler.

Correctness and cost

Both arms passed the same number of checkpoints cleanly. That number is dominated by a few of my own tests that were too brittle, which fired the same way in both arms, so I do not read much into the correctness magnitude from this run. I have since made those tests stricter, and the next run will give a correctness picture worth trusting. The early hint is that Spring broke a wider set of earlier rules.

Cost was close. The Spring chain cost about 79 US dollars in agent time. The OfficeFloor chain cost about 85. OfficeFloor did a little more total work, with more functions and more lines, because it spreads the same behaviour across more units. So it is not that one arm did less. It is that the two arms distributed the same job differently.

Honest limits

  • This is one chain per arm. Any single chain can go its own way. The real result needs many independent chains per arm and the confidence intervals across them. That run is underway.
  • The correctness comparison is muddied by brittle tests in this first chain. The structural comparison does not depend on those tests, so it stands.
  • The erosion ratio alone is a poor summary. The entry-handler complexity and the identity of the busiest function are what separate concentration from distribution.

Where this goes

On the measure that could not move in the first experiment, the two architectures now separate clearly, and in the predicted direction. Spring's create handler became a complexity-27 god method that is still growing. OfficeFloor's create function stayed flat at 9, with the complexity pushed out to bounded, single-purpose functions. The structural half of the thesis is looking well supported. The safety half, whether the composed design also regresses less, awaits the run on the stricter tests. When the full multi-chain run completes, the slopes and their confidence intervals will turn this first look into an answer.

No comments:

Post a Comment