This is a series on how software architecture shapes the way AI-written code decays. Last time we did something deliberately bad. We handed the AI the exact cost formula we were judging it by, and told it to keep the number low. It did. The number fell about forty-five fold. The code got worse. It scattered logic into tiny classes and copied code instead of reusing it. It optimised the number, not the design.
The obvious lesson was do not show the AI the number. So this run keeps the same number. But the AI never sees it. A tool sees it instead.
We gave the number to a tool, not the AI
The setup is the same as always. Two codebases that do the same REST service. Spring puts each new rule into one growing controller. OfficeFloor spreads each rule across many small wired functions. Ten independent runs on each side. Sixty change requests per run, one after another. Add a validation rule. Change how a field is stored. And so on for sixty steps.
The AI gets the plain request and nothing else. No formula. No cost. No hint that anything is being measured. It just implements the change.
Behind it sits the gate. On every change the gate scores the structural impact with the same formula as before. Here is that formula in plain terms.
cost = (complexity of the rest of the class) x (complexity of this method) x (lines changed)
The big term is the first one. Editing a method in a large class makes you carry the whole class in your head. So a change that piles more into an already heavy class scores high.
What the tool does when it fires
When a change scores over the line, the tool does not lecture the AI. It quietly runs one refactor first, on the clean code, before the change goes in. It tells the refactor step only the names of the classes that got too heavy. It does not say the cost. It does not say the formula. It does not say the words split into small classes. Those exact words are what caused the mess last time. It just asks for the code to be prepared so the next change lands in a way a maintainer would find natural.
Then the change is attempted again, on the cleaner code. The AI writing the feature never sees a score. There is nothing for it to optimise, so there is nothing for it to game. The number only ever points the tool at where to cut.
It worked on the change in front of it
The run finished clean. All twenty runs reached all sixty steps. Nothing broke the build. Nothing stopped early.
The tool fired on about a hundred and seventy Spring changes and about two dozen OfficeFloor changes. Spring trips far more often, which fits the whole series. Spring concentrates. OfficeFloor does not.
And when the tool fired, the change landed cleaner. The typical flagged Spring change dropped from an impact of about 11,800 to about 8,100. The typical flagged OfficeFloor change dropped from about 10,700 to about 5,100. So the mechanism is honest. The tool catches the change that would concentrate complexity, and moves that complexity somewhere less painful before the change goes in.
Then we looked at the standing code
A cheaper change on the day is not the point. The point is the shape of the code at the end. So we measured the heaviest class in the app, on the final code, the same way for every run. The AI could never aim at this number, because it never saw it.
| Spring's biggest class at the end | Weighted complexity |
|---|---|
| Plain spec, no gate | 135 |
| Told the formula | 22 |
| Hidden gate with refactor | 96 |
The hidden gate brings the god class down from 135 to 96. A real drop. Not a cure. The formula run shows 22, which looks far better. Hold that thought.
The 22 was a lie
The formula run scored beautifully because the AI gamed the score. So we counted what it actually built.
| Spring, per run | Plain spec | Told formula | Hidden gate |
|---|---|---|---|
| New classes made | 9 | 31 | 13 |
| Tiny static helpers | 2 | 21 | 5 |
| Duplicated lines | 2,230 | 2,510 | 2,230 |
The formula run tripled the class count. It made ten times as many tiny static helpers. A static helper in a small class costs almost nothing in the formula, so it is a cheap place to dump logic. It also duplicated the most code, because reuse means editing a class the formula was punishing. The god class number went down because the logic moved out of the controller and into a crowd of little files. A static helper costs you things the number does not see. No dependency injection. Harder to mock in a test. No place in a transaction.
The hidden gate did none of that. Its class count, its helper count, and its duplication all sit right next to the plain baseline. It lowered the real concentration by moving real structure. It did not dodge the number by hiding the complexity somewhere the number cannot look.
The total complexity never actually dropped
Here is the number that ties it together. We added up all the complexity that ended up in the code, wherever it lived.
| Spring, total complexity added | Amount |
|---|---|
| Plain spec, no gate | 282 |
| Told the formula | 268 |
| Hidden gate with refactor | 278 |
The total is about the same in all three. The work to build the feature does not shrink because you measured it. All any of these runs can change is where the complexity sits. The formula run shoved it into a pile of new files and called the job done. The hidden gate spread it a little more sensibly across the real code. Neither made it go away.
This is a very old idea
Fred Brooks split complexity into essential and accidental. The essential part is the difficulty of the problem. You cannot delete it. Larry Tesler said every system has an amount of complexity that cannot be removed. The only question is who carries it. That is exactly what the total above shows. And Goodhart said a measure that becomes a target stops being a good measure. That is exactly what the formula run showed. The hidden gate is the way to use a measure without making it a target.
Spring still grew a big class
Be clear about the limit. The hidden gate helps. It does not fix. Spring still ended with a class at 96 weighted complexity. That is still a big class. The tool lowers the impact of the change in front of it, but the complexity it pushes off today lands on a later change. A few Spring runs still ran away to a full god class anyway. And the gate did not make the code safer. It delivered a touch less and broke a touch more than the plain baseline. The win is structural. It is not free.
What to take from this
- Do not hand an AI the metric you are judging it by. It will optimise the metric and not the code.
- A metric is still useful. Let a tool read it and spend it. Keep it away from the thing writing the code.
- When you do spend it, aim a refactor at the sore spot. Name the heavy class. Do not prescribe the fix.
- Do not expect the complexity to vanish. Expect to choose where it lives.
What we are not claiming
This is one model, one kind of task, two codebases. The gate here records and continues. It does not throw changes away. That is a different test. A fourth run is going now. It gives the AI plain English advice to write good structure, with no number at all. That will tell us whether a tool beats simply asking nicely. Until then, the honest summary is small. Keep the number for the tool. Keep it away from the AI. Point it at the sore spot and let it cut.
No comments:
Post a Comment