Thursday, 3 September 2026

We Gave the Agent the Metric

A series on how software architecture shapes AI-driven code degradation. This post is a mid-run note. The run is still going, so there are no numbers here yet, only what we can see so far.

Every result in this series has an obvious rebuttal. Of course Spring's handler grows. Nobody told the agent not to let it.

So we told it. The current run hands the agent the exact cost formula the experiment measures. Not vague advice about clean code. The actual arithmetic. Take the complexity of the function you touch. Multiply it by the complexity of everything else in its class. Multiply that by how much of the function you change. Multiply the total by how many files you spread the change across. Lower is better. That is the whole objective, stated plainly, at every checkpoint.

There is a gate behind it as well. Each change is scored, and a change that lands above the threshold is thrown away and re-attempted after a refactor. That was the part we expected to do the work.

The gate has had nothing to do

It has not fired once.

The prompt alone was enough. Told what the measure is, the agent simply writes code that scores well on it, from the first attempt, nearly every time. The control loop we built to force the issue has been sitting idle while the prompt does all of it.

That is the first thing worth saying out loud. If you want an agent to optimise something, telling it the formula works.

Spring's god method never appeared

In every previous run, Spring's create-owner handler grew. Rule after rule landed in the same method until it was the largest thing in the codebase. That is the result the series has been built on.

This run, it does not happen. The handler ends close to where it started. The concentration measures that used to separate the two architectures no longer separate them. On the headline numbers, Spring now looks as clean as OfficeFloor, and on one of them it looks cleaner.

Taken at face value, that is the objection landing. Prompt better, and architecture stops mattering.

Then we looked at the code

The rules did not get smaller. Nothing was simplified away.

It moved.

Out of the handler, and into whatever host had the least surrounding complexity to pay for. Request advice. Aspects. Servlet filters. Entity listeners. Bean validators. Small classes that exist to hold one rule and nothing else. The logic is all still there. It is just somewhere else now, in a lot of somewhere elses.

In one chain the handler is byte for byte the file we started with. Not one of the rules is visible at the endpoint. They are all interceptors. The framework calls them. The order they run in is decided by annotations scattered across many files.

The measure cannot see where the work went

This is the part that matters, and it is uncomfortable, because it is our own measure that broke.

The comprehension measure we are most proud of follows the calls. It starts at the endpoint and walks every method the request path reaches, so work pushed downstream still counts. That was the fix a reader pushed us into making, and it was the right fix.

It has a blind spot. If nothing calls the code, the walk never reaches it. An interceptor is invoked by the framework, not by the handler, so a rule that becomes an interceptor leaves the measurement entirely. Most of Spring's create-owner logic is now outside what that walk can see.

So the flattering number is not a comprehension win. A developer changing one rule still has to find it first, and finding it is now harder, not easier. The measure got quieter. The code base did not get simpler.

Every chain escapes differently

There is no single shape to this. Run the same sequence again and the agent picks another way out. One chain went all in on request advice. Another used aspects for nearly everything. Others extracted static helpers, or pushed rules into validators and entity callbacks, and kept a slim handler calling them in order.

They all score well. They are wildly different code bases. If you are choosing an architecture on the strength of a number like this, that variance is the warning.

OfficeFloor moved too

Less, but in the same direction, and this one surprised us.

OfficeFloor answers a new rule by wiring in a new function. That is its whole idiom, and it is the behaviour the formula should reward. But wiring a function means touching the wiring file as well as writing the class. The formula multiplies by the number of files you touch. So the cheapest move stops being the idiomatic one.

What we see is a drift towards plain helper classes that nothing wires, called from a function that already exists. And a drift towards folding small rules into the entry function itself. That function's class is nearly empty. Under the formula, that makes it almost free to grow. The entry function is now doing more than it used to.

The pressure is far weaker than it is on Spring. The architecture already sits near where the formula wants it to be. But the direction is the same, and it is away from the architecture's own way of doing things.

Something is breaking more often

The agent still implements the rule in front of it at the same rate it always did. That has not moved.

What has moved is the damage to rules nobody asked it to touch. Both architectures now break more of them than they did before. Our working explanation is ordering. Rules spread across many independent interception points still have to run in some sequence. That sequence is no longer written down anywhere. It is implied by annotations, bean order, and framework lifecycle. An agent adding a late rule cannot see the ordering it is joining.

We want more of the run before we lean on this one.

What we think this is

It is Goodhart. A measure became a target and stopped being a good measure.

We should be fair to the agent here, because this is not simple cheating. Some of what it did is genuinely better. There is no god method. The worst single function in the code base is smaller than it was in either architecture before. Those are real improvements and we will report them as such.

But the gap between what the metric says happened and what actually happened is very large. The metric says the problem is solved. The code says the same work was cut finer and hidden better. Both of those are true at once, and only one of them shows up in a table.

The honest reading is that we cannot use this metric as an optimisation target and as evidence at the same time. As evidence about untargeted code, it held up well. Pointed at as a goal, it collapsed inside a single run, without a single gate intervention, using nothing but a prompt.

The run has chains to go. The numbers, and the final call on what this does to the series, will follow when it finishes.