I took the metric away

The last two posts did one thing. I stopped telling the AI to write good code and gave it a number instead. The exact ImpactGate cost formula, straight in the prompt. A gate that threw away any change that concentrated too much complexity. A refactor step told which classes were heavy. Everything the AI needed to keep the code clean, made measurable.

It gamed it. Exactly the way Goodhart says it will.

What the number did

When a measure becomes a target, it stops being a good measure. The AI was told the cost function and told to minimise it. So it minimised the function, not the concentration.

The formula charges a method for the complexity of the other methods in its class. So the AI stopped putting logic in the existing classes. It scattered the work into tiny new classes where the surrounding cost is near zero. And because reuse means editing a class the formula already penalises, it copied code instead of reusing it. The number went down. The code did not get better. It got more classes, more duplication, and the same tangle wearing a smaller cost.

That is the Goodhart effect in one run. The metric was honest until it became the objective. Then the AI optimised the metric and left the real problem alone.

The deeper trap: the spec becomes the program

The obvious fix is to add more metrics. Charge for duplication. Charge for new classes that only exist to dodge the first charge. Close each hole as the AI finds it.

I do not think that ends well. It is a losing game, and it is a game with a name.

Fred Brooks split software into essential and accidental complexity. The essential part is the difficulty of the problem itself. You cannot delete it. You can only decide where it lives. Larry Tesler said the same thing more bluntly. His Law of Conservation of Complexity says every system has an irreducible amount of it. The only question is who carries it.

So when I pile more rules into the metric, I am not removing the complexity. I am moving it into the metric. Keep going and the metric has to describe every structural decision precisely enough for the AI to optimise against it. At that point the metric, plus the spec, is a full description of the system. It is the program. Written in a language that cannot be run, cannot be tested, and has no tools.

That is the trap. If keeping the code clean requires me to specify the system twice, once as a spec to optimise and once as the code it produces, then the spec is just a more expensive way to maintain the definition of the system than the code was. Code is already the most precise, testable, tooled description we have. Replacing it with an ever growing specification is not progress.

So I stopped adding to the number. I started taking it away. Two experiments, two ways to remove it.

Experiment one: keep the tool, hide the number

The first experiment keeps ImpactGate but never lets the AI see the formula.

The AI implements each change with a plain spec. No metric. No objective to optimise. ImpactGate still scores the change in the background. When a change concentrates too much, one refactor runs first. But the refactor is told only the symptom, not the cure.

It is given the names of the classes that ended up carrying too much. Locations only. Not the cost, not the formula, not the words "split into small classes". Those phrases are what drove the dispersal and the duplication last time. Here is the whole refactor prompt.

A change is about to be made to this codebase. When that change was
implemented directly, these classes ended up carrying more complexity
than they can hold comfortably:

{drivers}

Before the change is made, prepare the existing code so the change can
land as code a maintainer of this codebase would recognise as natural,
consistent with how this codebase already structures similar work.

It names the sore spot and asks for something natural. It does not hand over a lever to pull.

One more guard. The refactor's own new lines run through a deterministic quality gate. Duplicated lines and known bad patterns are caught by jscpd and ast-grep, not by a prompt. So the AI cannot buy a quieter structure with copy paste. The prompt asks for good judgement. The gate, not the prompt, forbids the cheap moves.

The question is simple. With no number to optimise and only a nudge at the sore spot, does the structure stay cohesive?

Experiment two: ask in plain English

The second experiment drops the tool entirely. No gate. No refactor step. No number anywhere.

I am not ruling out that guidance helps. I am ruling out the formula. So the ask goes back to English, but honest English. Not a target to optimise. A plain request for structure, in the words a senior engineer would use in review.

As you do, keep the code well structured: put each piece of logic where
it belongs, in a small unit with a single responsibility, and reuse
existing code instead of copying it. Do not let any one class or method
grow into a catch-all that accumulates unrelated logic.

That is the whole intervention. Same single turn as the plain spec run. Same everything else. Only the wording changes. It says nothing about a metric, a cost, or the experiment. It cannot be gamed, because there is no number to game. It also cannot be checked, by the AI or by me, which is exactly the weakness the formula was meant to fix.

So this experiment asks the honest version of the original question. Not "can a measurable target keep code clean", which the AI answered by gaming the target. But "can plain good advice keep code clean", with nothing to optimise against it.

What the two experiments are really testing

Line everything up and there is a clean ladder.

• Plain spec. No help at all. The baseline erosion.
• Plain English. Good advice, no number. The prompting lever on its own.
• Refactor run. A tool that catches the sore spot and nudges it, still no number shown. The tool lever on its own.
• The formula. The full measurable target. Already run. Already gamed.

Each step adds exactly one thing. So the gap between two steps is the effect of that one thing. And every one of them is measured the same way at the end, on the shape of the standing code, not on any score the AI could aim at.

Here is the worry that started all this. Tesler says the complexity never leaves. It only moves. My first bet was to hold it down with a measurement, and the AI gamed the measurement. These two experiments try the opposite and take the number away.

But the complexity still has to land somewhere. Maybe the AI invents its own target to chase, and games that instead, with no formula in sight. Maybe the only way to stop it is to keep adding to the spec, until the spec is the program again. If either happens, then a metric was never the right lever. A better number cannot fix a problem that lives in the architecture. The fix would have to be the architecture itself.

That is what these two experiments are built to find out. Same harness, same problem, same chains, same measurement at the end.

The runs are underway.