The Number the AI Never Saw: We Spent the Change Impact With a Tool

This is a series on how software architecture shapes the way AI-written code decays. Last time we did something deliberately bad. We handed the AI the exact cost formula we were judging it by, and told it to keep the number low. It did. The number fell about forty-five fold. The code got worse. It scattered logic into tiny classes and copied code instead of reusing it. It optimised the number, not the design.

The obvious lesson was do not show the AI the number. So this run keeps the same number. But the AI never sees it. A tool sees it instead.

We gave the number to a tool, not the AI

The setup is the same as always. Two codebases that do the same REST service. Spring puts each new rule into one growing controller. OfficeFloor spreads each rule across many small wired functions. Ten independent runs on each side. Sixty change requests per run, one after another. Add a validation rule. Change how a field is stored. And so on for sixty steps.

The AI gets the plain request and nothing else. No formula. No cost. No hint that anything is being measured. It just implements the change.

Behind it sits the gate. On every change the gate scores the structural impact with the same formula as before. Here is that formula in plain terms.

cost = (complexity of the rest of the class) x (complexity of this method) x (lines changed)

The big term is the first one. Editing a method in a large class makes you carry the whole class in your head. So a change that piles more into an already heavy class scores high.

What the tool does when it fires

When a change scores over the line, the tool does not lecture the AI. It quietly runs one refactor first, on the clean code, before the change goes in. It tells the refactor step only the names of the classes that got too heavy. It does not say the cost. It does not say the formula. It does not say the words split into small classes. Those exact words are what caused the mess last time. It just asks for the code to be prepared so the next change lands in a way a maintainer would find natural.

Then the change is attempted again, on the cleaner code. The AI writing the feature never sees a score. There is nothing for it to optimise, so there is nothing for it to game. The number only ever points the tool at where to cut.

It worked on the change in front of it

The run finished clean. All twenty runs reached all sixty steps. Nothing broke the build. Nothing stopped early.

The tool fired on about a hundred and seventy Spring changes and about two dozen OfficeFloor changes. Spring trips far more often, which fits the whole series. Spring concentrates. OfficeFloor does not.

And when the tool fired, the change landed cleaner. The typical flagged Spring change dropped from an impact of about 11,800 to about 8,100. The typical flagged OfficeFloor change dropped from about 10,700 to about 5,100. So the mechanism is honest. The tool catches the change that would concentrate complexity, and moves that complexity somewhere less painful before the change goes in.

Then we looked at the standing code

A cheaper change on the day is not the point. The point is the shape of the code at the end. So we measured the heaviest class in the app, on the final code, the same way for every run. The AI could never aim at this number, because it never saw it.

Spring's biggest class at the end	Weighted complexity
Plain spec, no gate	135
Told the formula	22
Hidden gate with refactor	96

The hidden gate brings the god class down from 135 to 96. A real drop. Not a cure. The formula run shows 22, which looks far better. Hold that thought.

The 22 was a lie

The formula run scored beautifully because the AI gamed the score. So we counted what it actually built.

Spring, per run	Plain spec	Told formula	Hidden gate
New classes made	9	31	13
Tiny static helpers	2	21	5
Duplicated lines	2,230	2,510	2,230

The formula run tripled the class count. It made ten times as many tiny static helpers. A static helper in a small class costs almost nothing in the formula, so it is a cheap place to dump logic. It also duplicated the most code, because reuse means editing a class the formula was punishing. The god class number went down because the logic moved out of the controller and into a crowd of little files. A static helper costs you things the number does not see. No dependency injection. Harder to mock in a test. No place in a transaction.

The hidden gate did none of that. Its class count, its helper count, and its duplication all sit right next to the plain baseline. It lowered the real concentration by moving real structure. It did not dodge the number by hiding the complexity somewhere the number cannot look.

The total complexity never actually dropped

Here is the number that ties it together. We added up all the complexity that ended up in the code, wherever it lived.

Spring, total complexity added	Amount
Plain spec, no gate	282
Told the formula	268
Hidden gate with refactor	278

The total is about the same in all three. The work to build the feature does not shrink because you measured it. All any of these runs can change is where the complexity sits. The formula run shoved it into a pile of new files and called the job done. The hidden gate spread it a little more sensibly across the real code. Neither made it go away.

This is a very old idea

Fred Brooks split complexity into essential and accidental. The essential part is the difficulty of the problem. You cannot delete it. Larry Tesler said every system has an amount of complexity that cannot be removed. The only question is who carries it. That is exactly what the total above shows. And Goodhart said a measure that becomes a target stops being a good measure. That is exactly what the formula run showed. The hidden gate is the way to use a measure without making it a target.

Spring still grew a big class

Be clear about the limit. The hidden gate helps. It does not fix. Spring still ended with a class at 96 weighted complexity. That is still a big class. The tool lowers the impact of the change in front of it, but the complexity it pushes off today lands on a later change. A few Spring runs still ran away to a full god class anyway. And the gate did not make the code safer. It delivered a touch less and broke a touch more than the plain baseline. The win is structural. It is not free.

What to take from this

• Do not hand an AI the metric you are judging it by. It will optimise the metric and not the code.
• A metric is still useful. Let a tool read it and spend it. Keep it away from the thing writing the code.
• When you do spend it, aim a refactor at the sore spot. Name the heavy class. Do not prescribe the fix.
• Do not expect the complexity to vanish. Expect to choose where it lives.

What we are not claiming

This is one model, one kind of task, two codebases. The gate here records and continues. It does not throw changes away. That is a different test. A fourth run is going now. It gives the AI plain English advice to write good structure, with no number at all. That will tell us whether a tool beats simply asking nicely. Until then, the honest summary is small. Keep the number for the tool. Keep it away from the AI. Point it at the sore spot and let it cut.

Preprint / cs.SE / Empirical Software Engineering

Spending the Cost Function Without Telling the Agent

Two architectures. Twenty chains. Sixty accumulating rules each. A structural metric read by a tool instead of handed to the agent.

Daniel Sagenschneider

OfficeFloor · independent research · blog.officefloor.net

September 2026 · correspondence: daniel@officefloor.net

---

Abstract

Change impact scores a code change by the complexity it disturbs. A prior study in this series disclosed that cost function to a coding agent as its objective. The agent optimised the number and not the code. That was Goodhart's law, and it held cleanly. This paper keeps the same cost function but never shows it to the agent. The agent receives a plain specification. A tool scores each change in the background. When a change concentrates too much complexity, the tool runs one refactor first, on the clean code, and names only the heavy classes.

Across two architectures and twenty independent chains of sixty accumulating changes, the hidden gate lowers the rate of structural concentration by a real amount. Spring's per-checkpoint impact slope falls from 435 to 127, a cut of about seventy percent, with non-overlapping intervals. It does so without the class proliferation and duplication the disclosed formula produced. The disclosed run reported an impact slope of 9.7 for Spring, far lower, yet it tripled the class count, multiplied static utilities nine fold, and broke the most behaviour.

The total complexity that lands in the codebase is about the same in all three conditions. Only its distribution moves. Disclosure moved it to where the measure was blind. The hidden gate moved it in the code. The reduction is partial, the functional cost is small, and Tesler's conservation of complexity still holds. The metric works as an instrument a tool spends. It does not work as a target handed to the writer.

Keywords: Goodhart's law · conservation of complexity · AI-assisted development · software architecture · code degradation · refactoring · cyclomatic complexity

1Introduction

A prior post handed the agent the exact cost function and told it to minimise it. The number fell by about forty-five fold. The code did not improve. The agent scattered logic into tiny classes where the surrounding cost is near zero. It copied code rather than edit a class the formula already penalised. The concentration the metric was meant to prevent simply moved to where the metric was not looking.

This raises a narrower question. The failure above was disclosure, not the metric. So can the same cost function reduce concentration if the agent never sees it, and a tool spends it instead. The intervention here is a gate that reads the score and acts on it, with a wall between the tool and the agent. The agent writes the code. The tool watches and, when needed, prepares the ground.

2Background and related work

Brooks separated essential complexity, the difficulty of the problem, from accidental complexity, the part we add [1]. Essential complexity cannot be deleted by better instructions. Tesler's law of conservation of complexity says every system carries an irreducible amount [2]. The only open question is who holds it. Goodhart's law says a measure that becomes a target stops being a good measure [3]. The disclosed run was a direct demonstration of all three at once.

The structural measures are standard. Cyclomatic complexity follows McCabe [4]. Weighted methods per class follows Chidamber and Kemerer [5]. The cost function combines them to price the context a change must disturb [6].

3The metric and the intervention

For every function a change touches, the cost is:

cost = max(WMC_other, 1) × CC × max(1, changed_lines)

CC is that function's cyclomatic complexity. WMC_other is the summed complexity of the other methods in its class, the context a maintainer must hold to edit it safely. changed_lines is how many of its lines the change adds or edits. The costs are summed over every changed function and multiplied by the number of files touched. The dominant term is WMC_other. A method inside a large class pays for the whole class.

The gate scores each change against a fixed percentile of a reference distribution. A change over the line is flagged. On a flag the tool does not lecture the agent. It runs one refactor step on the clean pre-change code. The refactor prompt names the heavy classes and asks for a natural restructuring. It never states the cost, the formula, or the phrase "split into small classes". Those phrases drove the dispersal and duplication in the disclosed run.

A change is about to be made to this codebase. When that change was
implemented directly, these classes ended up carrying more complexity
than they can hold comfortably:

{drivers}

Before the change is made, prepare the existing code so the change can
land as code a maintainer of this codebase would recognise as natural,
consistent with how this codebase already structures similar work.

A deterministic quality gate checks the refactor's own added lines for duplication before the change is re-attempted. The gate is advisory. It records the outcome and continues. A flag never aborts a chain. The number only ever points the tool at where to cut.

4Study design

Two codebases implement the same REST service. Spring routes each new rule through one growing controller. OfficeFloor spreads each rule across many small wired functions. The same total complexity lands in both, but Spring concentrates it. This is the independent variable the whole series holds.

Each arm runs ten independent chains. Each chain applies sixty sequential change requests to the service. Every arm gets the same specifications and the same plain implement prompt. The agent is the same model throughout. The gate threshold is a shared cutoff, calibrated to OfficeFloor's own cohesion, so OfficeFloor trips rarely and Spring trips often. All structural measures are recomputed on the standing code at each checkpoint, the same way for every condition, on numbers the agent could never aim at. Slopes are ordinary least squares of the metric on checkpoint index. Confidence intervals are ninety-five percent, bootstrapped over chains.

Three conditions are compared. Control is the plain specification with no gate. Disclosed is the earlier run with the formula in the prompt and a hard gate. Refactor-only is the hidden gate described above. A fourth condition, plain English advice with no number, is still running and is not reported here.

5Results

The refactor-only run completed cleanly. All twenty chains reached the final checkpoint. No build broke. No chain stopped.

When the tool fires, it works on the change in front of it. The refactor triggered on 171 of 600 Spring checkpoints and 24 of 600 OfficeFloor checkpoints. On a flagged change the accepted impact is lower than the first attempt. The median flagged Spring change falls from about 11,800 to about 8,100. The median flagged OfficeFloor change falls from about 10,700 to about 5,100. So the per-change mechanism is honest. Spring trips the gate far more often, which is the concentrated versus cohesive split the series keeps finding.

The result that matters is the standing code. Table 1 gives the rate at which the scored impact of each new change grows over a chain.

Change-impact slope per checkpoint	Control	Disclosed	Refactor-only
Spring	435	9.7	127
OfficeFloor	76	5.7	28

Table 1. Ordinary-least-squares slope of change impact on checkpoint index. Lower is slower concentration. Spring refactor-only 127 [76, 179] versus control 435 [307, 587]: the intervals do not overlap.

Table 2 gives the standing god class. This is the weighted complexity of the class the endpoint routes through in both arms, measured on the final code, and its growth per checkpoint.

Handler class, Spring	Control	Disclosed	Refactor-only
Final weighted complexity	135	22.5	96
Growth per checkpoint	1.77	0.04	1.17

Table 2. The hidden gate dents the god class but does not cure it. Spring still ends with a single class at about 96 weighted complexity, below the control's 135 and far above the disclosed run's 22.

The disclosed number looks like a win until you look at where the code went. Table 3 is the gaming signature on Spring, the arm with a monolith to dodge.

Spring, per chain unless noted	Control	Disclosed	Refactor-only
New classes created	9	31.5	13.3
Of those, static utilities	2.3	20.6	5.2
Duplicated lines at tip	2,230	2,510	2,230
Cumulative CC, in new files	46	218	99
Cumulative CC, in existing files	236	50	178
Cumulative CC, total	282	268	278

Table 3. Read the last row first. The total complexity is about the same in all three. Only its distribution moves. The disclosed run inverted where the complexity lives and raised duplication. The refactor-only run sits at or near the control on every row.

Table 4 gives the cost. EvoScore is a functional delivery score over the chain. True regressions count previously passing behaviour the agent later broke, excluding intended changes.

Spring	Control	Disclosed	Refactor-only
EvoScore	0.787	0.440	0.712
True regressions	37	56	41

Table 4. The disclosed run was also the worst at the task. The refactor-only run stays close to the control. The structural gain does not come with a safety gain.

6Discussion

The refactor-only gate gives a real reduction in concentration, and an honest one. The standing evidence shows structure moved, not a number gamed. But the reduction is partial. The per-change cut is reliable, yet the standing curve bends only part way. The complexity the tool pushes off today's change lands on a later one. That is conservation again. A gate on each change slows the concentration on the arm that concentrates. It has not stopped it.

The contrast with disclosure is the core finding. Disclosure produced a far lower score and worse code. The hidden gate produced a higher score and better-shaped code. The same cost function, read by a tool rather than chased by the writer, changes the structure instead of the number.

7Threats to validity

One model, one task family, one pair of architectures. The gate here is advisory, so it records and continues rather than discarding a change. A hard gate is a separate condition. The threshold is calibrated to OfficeFloor's cohesion, which fixes how often each arm trips. Ten chains per arm leave real variance, and Spring's variance is wide. Some chains still run away to a full god class. The smell half of the duplication and pattern detector contributed a negligible number of lines, so the duplication figure is effectively a clone measure. EvoScore and true regressions are measured independently of the structural score, which is why they can disagree with it.

8Reproducibility and data availability

Availability

Each chain is a git branch of sequential checkpoint commits. The analysis recomputes every structural measure from the commits, so no derived value is read back. Each run pins its own configuration snapshot, so the metrics match how that run was scored. Confidence intervals are bootstrapped over chains.

9Conclusion

Brooks's essential complexity did not shrink under a plain spec, a hidden gate, or a disclosed formula. Tesler's conservation held in every condition. The total complexity landed in roughly the same amount each time. When the measure was disclosed, the work moved to wherever the measure was not looking, and the code got worse. When the measure was kept back and spent by a tool, the work moved in the code, and the concentration fell part way with no gaming. Use the metric to watch and to aim a refactor. Do not hand it to the writer as the goal.

---

1. F. P. Brooks. No Silver Bullet: Essence and Accidents of Software Engineering. Computer, 1987.
2. L. Tesler. The Law of Conservation of Complexity. Mid-1980s.
3. C. A. E. Goodhart. Problems of Monetary Management: The UK Experience. 1975.
4. T. J. McCabe. A Complexity Measure. IEEE Transactions on Software Engineering, 1976.
5. S. R. Chidamber, C. F. Kemerer. A Metrics Suite for Object Oriented Design. IEEE Transactions on Software Engineering, 1994.
6. D. Sagenschneider. Telling the Agent the Cost Function. blog.officefloor.net, September 2026.

Slopes are OLS on checkpoint index · intervals 95% bootstrapped over chains · ten chains per arm · sixty checkpoints per chain

The refactor run halfway

The last post set up a ladder. Four runs, each adding exactly one thing. Plain spec, the baseline erosion. Plain English, good advice with no number. The refactor run, a tool that catches the sore spot and nudges it, still no number shown. The formula, the full measurable target, already run and already gamed.

This is an update on the third rung. The refactor run is about halfway through, and it is behaving. So here is what I can already see, and what I still cannot.

What the refactor run is

It is the tool lever on its own. The agent gets the plain spec and nothing else. No formula, no cost, no mention of the experiment. It just implements the change.

Behind it sits ImpactGate. On every change it scores the structural impact with the same cost formula from the earlier posts. When a change scores above the line, the tool does not lecture the agent. It runs a refactor step on the clean code, names the heavy classes, and breaks them into smaller cohesive ones. Then the change is attempted again on the cleaner code.

The key difference from the formula run is the wall between the tool and the agent. The agent never sees the number. There is nothing for it to optimise, so there is nothing for it to game. The number only ever points the tool at where to cut.

How far it has run

Two codebases, Spring and OfficeFloor, the same two as always. Ten independent chains each, sixty checkpoints a chain. Six chains are done on each side, and the seventh is running. So a little over half the run is in.

It is clean so far. Every finished chain ran all sixty checkpoints. No build ever broke. No chain stopped early. The gate is advisory here, so it records and continues rather than aborting, which is why a flag never derails a chain.

The one thing I can already see

When the tool fires, it works.

The refactor has triggered on about a hundred and twenty checkpoints so far. Every one of them was a change the formula scored near the top of the scale before the cut. After the refactor, the same change lands on cleaner code and its impact score falls by about two thirds. The average impact on those changes drops from roughly nineteen thousand to roughly six thousand. It falls in about nine of every ten cases. Spring trips the gate far more often than OfficeFloor, which is exactly the concentrated-versus-cohesive split the whole series keeps finding.

So the mechanism is honest. The tool catches the change that would concentrate complexity, and the refactor moves that complexity somewhere less painful before the change lands. No number was ever shown to the thing writing the code.

The curve it has to bend

A cheaper change on the day is not the point of the experiment. The point is the shape of the standing code at the end. So the number that matters is the heaviest class in the app, measured by its weighted complexity, and how fast that grows checkpoint after checkpoint. That is a god class forming in slow motion. It is measured on the final code, the same way for every rung, on a number the agent could never aim at.

On the plain spec baseline the biggest class keeps growing. Spring's heaviest class gains about one and eight tenths of weighted complexity every checkpoint. OfficeFloor's gains about seven tenths. Spring concentrates roughly two and a half times faster. That gap is the erosion the tool is meant to fight.

An early read of the slope

I did not want to quote a number off half a run. But half a run with nothing in it says very little, so here is the honest interim, with the caveats loud. Six of the ten chains are in on each side. Measured the same way as the baseline, the growth per checkpoint so far is:

• OfficeFloor. Baseline about seven tenths. Refactor run about six and a half tenths. Essentially unchanged.
• Spring. Baseline about one and eight tenths. Refactor run about one and a half. Lower, but the chains are all over the place.

OfficeFloor is the easy read. It was already cohesive, so the tool rarely fires and there is little to bend. Its slope barely moves, which is what you would expect when the problem was never there.

Spring is the interesting one, and not in the clean way I hoped. The average slope drops by about a fifth. But the spread is wide. Four of the six Spring chains stayed reasonably flat. Two of them still ran away, ending with a single class carrying about a hundred and sixty weighted complexity, a full god class, gate and refactor notwithstanding.

So the per-change cut is real, and it is not reliably reaching the end state. The tool lowers the impact of the change in front of it. The standing god class on the arm that has the problem still grows at roughly four fifths of the baseline rate, and unevenly. The complexity the tool pushed off today's change is landing on some later one.

Treat those numbers as a direction, not a verdict. They are six chains, not ten. The final figure uses a more careful slope with confidence bands, and Spring's variance is exactly the kind that moves once the last chains land. The sign looks right. The size is not settled.

Where this leaves the ladder

The formula run proved a measurable target gets gamed. This run is the opposite bet. Keep the number away from the agent, and let a tool spend it instead.

Halfway in, the tool does its job on every change it touches, and the standing curve bends only part way. That is the Tesler shape again. The complexity did not leave. It moved. A gate on each change slows the concentration on the arm that concentrates, but it has not stopped it, and twice in six tries it barely dented it.

A few more days and both sides finish. Then the real measurement, on the standing code.

We told the AI about the Change Impact

A series on how software architecture shapes AI-driven code degradation. This is the plain version of a result. The paper has the numbers and the caveats.

We have been running an experiment. An AI agent gets sixty change requests, one after another, all landing on the same REST endpoint. Add a validation rule. Change how phone numbers are stored. Add a duplicate check. Sixty of them, in order, with the full test suite run after each one.

We do this twice. Once on a normal Spring codebase. Once on the same application built with OfficeFloor. Then we look at what sixty changes did to each one.

The Spring result has been the same every time. The create-owner handler method grows. Rule after rule lands in it. By the end it is the biggest thing in the codebase and every new rule means opening the same enormous method again.

And there has always been an obvious objection to that. Nobody told the AI not to do it.

So we told it

We have a metric called change impact. It scores a change by how much complexity it disturbs, rather than by how many lines it edits. Touching a small method in a small class is cheap. Adding ten lines to a huge method in a huge class is expensive.

Roughly, for every method you change:

cost = (how heavy the class already is)
     x (how complicated the method is)
     x (how many of its lines you changed)

Add that up for every method you touched. Multiply by the number of files. Lower is better.

This time we put that formula in the prompt. Every single change request. We told the AI exactly how it was being scored, and we told it which part of the formula mattered most.

It worked immediately

We had also built a safety net. If a change scored badly, we would throw it away and make the AI refactor first. We expected that to be doing the work.

It fired three times. Out of twelve hundred changes.

The prompt alone was enough. Told what the score was, the AI wrote code that scored well on it, first try, nearly every time.

And the god method never appeared. Here is how much the controller file grew over sixty rules:

Spring controller, lines added over 60 rules	Ten runs
Normal prompt	604 to 962
Told the formula	2 to 44

That is the objection landing. Prompt better and the problem goes away. If we stopped here, this post would say architecture does not matter much and good instructions do.

Then we looked at where the code went

We have a second check that does not care about any of our metrics. It takes the finished codebase, finds every line the run changed, and works out which method that line ended up in. Then it adds up how complicated all those methods are. It is deliberately dumb. It just asks how much logic now exists and where it lives.

Spring, after 60 rules	Normal prompt	Told the formula
Total logic written	282	268
...sitting in brand new files	46	218
...sitting in files that already existed	236	50
Number of files involved	19	41

Look at the first row. The amount of logic is the same. 282 before, 268 after. Nothing got simpler.

Now look at the next two rows. It all moved. Out of the files that existed, into files the AI created. Twice as many files.

This is not the AI being clever or sneaky. Look at the formula again. The first term is how heavy the class already is. A brand new file has nothing in it. So that term is as small as it can possibly get. If you want a low score, the cheapest thing you can do is put your code somewhere nothing else lives.

So it did. Sixty times.

What thirty-one new classes look like

Per run, the normal prompt created about nine new classes. Told the formula, it created about thirty-one. Roughly twenty of those were static utility classes. Classes with one static method, holding one rule, and nothing else.

On the score, that is perfect. A static method in an otherwise empty class has almost no surrounding complexity to pay for.

In a real Spring codebase, it costs you things a junior engineer runs into fast. A static method is not a bean. You cannot inject anything into it. You cannot swap it out in a test. Spring cannot wrap it in a transaction or a proxy. You have made the metric happy and given up most of what the framework is for.

The duplication went up too, and for a reason worth understanding. Reusing an existing helper means adding a line to a method in a class that already has weight. The formula charges you for that. Writing your own copy in a fresh file is free. So the AI wrote its own copy. Duplicated lines went from about 2230 to about 2510, in a codebase that had got smaller overall.

One run vanished completely

We ran ten independent Spring chains. Most of them dispersed into helper classes as described. Two did something else.

In one of them, the create-owner handler at the end of sixty rules is the method we started with. Map the request. Save the owner. Set the location header. Return 201. That is it.

Every one of the sixty rules is a @RestControllerAdvice class. Eighteen of them. They run before the handler is ever called, because Spring invokes them, not the handler.

Our comprehension metric follows method calls from the endpoint. It is a good metric. It was added because a reader correctly pointed out that a pipeline can hide work in its later stages, and following the calls fixes that.

It cannot follow something nothing calls. An interceptor is invoked by the framework. So for that run, our metric reports the create path as having a complexity of 3, for a codebase implementing sixty business rules.

That is not a clean codebase. If you are asked to change the phone number rule, you still have to find it first, and finding it just got much harder. The number got quieter. The code did not get simpler.

The part that actually matters

All of the above is arguing about metrics. This part is not.

Spring	Normal prompt	Told the formula
Implemented the rule it was asked for	every time	every time
Whole test suite still green	79% of changes	44% of changes
First rule permanently broken at	rule 47	rule 24

The AI still did the job in front of it. Every time. What it stopped doing was keeping the previous fifty-nine rules working.

Something broke, earlier and more often, and none of the structural metrics showed it. The tests showed it.

We had a theory. Rules scattered across separate interceptors still have to run in some order, that order is no longer written down anywhere, and an AI adding rule 40 cannot see the ordering it is joining. It is a nice theory. The finished data does not support it. The runs that leaned hardest on interceptors actually broke slightly less. So we do not know why yet, and we are saying so rather than keeping a tidy explanation that the numbers disagree with.

This is a very old idea

Two of them, actually.

Fred Brooks, in 1986, split the difficulty of software into two parts. Essential complexity is the problem itself. Sixty business rules are sixty business rules. Accidental complexity is the mess we add on top through how we choose to build it. His argument was that no tool removes the essential part.

That is exactly the first row of our second table. 282 before, 268 after. The rules are the rules. No prompt made them cheaper.

Larry Tesler put it a different way in the 1980s, usually called the law of conservation of complexity. Complexity does not disappear. It moves. Design decides who has to deal with it, not whether anybody does.

That is the rest of the table. The complexity moved out of the handler and into forty-one files, and in two runs it moved somewhere our tooling could not follow at all.

And the reason it moved is Goodhart's law, in its usual form: when a measure becomes a target, it stops being a good measure. We knew that. We still did not expect it to happen this completely, in one run, from one paragraph of prompt, with no gate ever firing.

What to take from this

If you are early in your career and working with AI tools, this is the practical version.

A green metric is not the same as good code. When a number improves a lot and quickly, ask what moved. Not what got deleted. Things rarely get deleted.

Be suspicious of a class that exists to hold one rule and nothing else. It is often a real improvement. It is also the cheapest way to make almost any code metric look better, so it is worth checking which one you are looking at.

Static helpers look free and are not. You give up injection, mocking, transactions and proxying. If someone, human or AI, is producing a lot of them quickly, that is worth a conversation.

If you cannot find where a rule runs, the codebase got harder, whatever the dashboard says. Being able to open one method and read what happens is worth a lot.

And keep your tests. In this experiment every structural metric we own said the code got better. The acceptance suite was the only thing that noticed rules quietly breaking from change 24 onwards. That is not a small detail. It is the whole reason we caught this.

What we are not claiming

We told the AI to optimise a formula and things got worse. We have not yet shown that a normal request for good structure would do the same. There is a difference between "here is the arithmetic, minimise it" and "please keep this code well organised", and we are running that second version now.

Until that finishes, the honest claim is narrow. Do not hand an AI the metric you are judging it by. Use the metric to watch. Do not use it as the goal. Those are two different jobs and it can only do one of them.

Preprint / cs.SE / Empirical Software Engineering

Telling the Agent the Cost Function

Two architectures. Twenty chains. Sixty accumulating rules each. What happens to a codebase when its structural metric becomes the objective.

Daniel Sagenschneider

OfficeFloor · independent research · blog.officefloor.net

September 2026 · correspondence: daniel@officefloor.net

---

Abstract

Change impact scores a code change by the complexity it disturbs [6]. It was defined inside a controlled degradation study. There it tracked how a codebase erodes as an AI agent lands accumulating changes on one endpoint. This paper reports what happens when that score is handed to the agent as its objective. Ten independent chains per architecture. Sixty accumulating rules per chain. The same endpoint and the same acceptance suite as an untouched control.

Disclosure works on the disclosed measure. The per-checkpoint impact slope falls 45-fold for Spring. It falls 13-fold for OfficeFloor. Spring's growing handler does not appear. That handler is the central finding of the prior work in this series. Its controller file grows by 604 to 962 lines across the ten control chains. It grows by 2 to 44 lines across the ten disclosed chains. Two of the three concentration statistics stop separating the architectures at all.

The complexity did not leave. We attributed every changed line at the chain tip to the function that now contains it. The total cyclomatic complexity Spring's run touched is 282 before and 268 after. That is unchanged inside its spread. The share living in files the run created rises from 46 to 218. The file count doubles. The rules were relocated. They were not removed. This is Brooks's essential complexity [3], conserved as Tesler described [4]. What rose is the accidental part. Created classes per Spring chain go from 9.0 to 31.5. Two-thirds of them are static utilities. Duplicated lines rise on a codebase that shrank. In two of ten chains the rules moved into framework-dispatched interceptors. Those rules left the call-graph measurement entirely. One chain reports a create-path complexity of 3 with all sixty rules implemented.

Delivered correctness fell. Every checkpoint's own new rule still landed in every chain. func is 1.000 throughout. Retention of previously passing rules dropped from 0.787 to 0.440 for Spring. It dropped from 0.732 to 0.577 for OfficeFloor. The first standing failure arrives at rule 24 instead of the high forties. We report the mechanism as unexplained. The ordering hypothesis we proposed mid-run is not supported by the completed data. The conclusion is narrow and firm. Change impact is usable as evidence about code that is not targeting it. It is not usable as an optimisation target.

Keywords: change impact · Goodhart's law · conservation of complexity · AI-assisted development · software architecture · code degradation · metric gaming · cyclomatic complexity

1Introduction

Every result in this series has had the same rebuttal waiting for it. Spring's request handler accumulates rule after rule. It ends as the largest thing in the codebase. OfficeFloor's wired pipeline stays flat. Of course it does. Nobody told the agent not to let it happen.

So we told it. This paper reports a full sweep in which the agent is handed the exact arithmetic its work will be scored by. It is handed it at every one of sixty checkpoints. Not advice about clean code. The cost function itself, with its dominant term named.

The question is not whether an agent can optimise a disclosed objective. It can. The first result below is how completely. The real question is what optimising it does to the code underneath. We measure that with instruments the agent was never told about. An unscoped audit of where complexity physically ended up. A classification of what kind of class now holds each rule. A duplication detector. The acceptance suite.

2Background and related work

That an optimised measure stops measuring is old. Goodhart observed it for monetary policy. Strathern's restatement is the one usually quoted [5]. When a measure becomes a target, it ceases to be a good measure. The software-metrics literature has its own long record of this. Lines-of-code targets are the familiar case. What is new here is the speed, and the operator. An AI agent given a formula optimises it immediately. It does so at every checkpoint. It does not tire and it does not negotiate. So the failure mode becomes observable inside a single controlled run. It no longer needs quarters of organisational drift to show up.

Two older results frame what we found underneath. Brooks divided the difficulty of software in two [3]. Essence is the complexity inherent in the problem being solved. Accident is the complexity introduced by how we happen to build it. He argued that no tooling improvement removes the essential part. Tesler's law of conservation of complexity makes a related point about placement rather than tooling [4]. A given task carries an irreducible amount of complexity. Design decides who bears it. Design does not decide whether it exists. Tesler's framing is usually applied across the user and developer boundary. Our result is an instance of it inside a single codebase. Sixty business rules are essential complexity. The formula moved them. It did not remove them. The accidental part grew.

The experiment's measures come from two sources. SlopCodeBench supplies erosion, verbosity and degradation slope [1]. SWE-CI supplies normalized change, EvoScore and zero-regression rate [2]. The change-impact score itself is defined and validated elsewhere [6]. That validation tests it against defects in human-written code.

3The metric and the intervention

Change impact charges a change by the complexity it disturbs. It does not charge by the lines it edits. For each function the change adds lines to:

cost(function) = max(WMC_other, 1) × CC × max(1, Δlines)

CC is the changed function's own cyclomatic complexity. Δlines is how many of its lines were added or edited. WMC_other is the summed complexity of the other methods in its class. It is measured on the state before the change. It stands for the surrounding context a maintainer must hold to edit the function safely. The per-function costs are summed. The total is then multiplied by the number of files touched. The full definition and rationale are in [6].

The intervention is simple. That text goes into the implement prompt at every checkpoint. The dominant term is named outright:

Write it to keep structural impact low. Structural impact is a
measurable cost, not a vague goal. For every function you change,
the cost is:
    cost = max(WMC_other, 1) * CC * max(1, changed_lines)
...
So the biggest lever is WMC_other: a method inside a large class
pays for the whole class. Do not grow one method or one class.

The control prompt is the instruction used unchanged throughout this series. Implement the specification. Make the tests pass. Nothing else differs between the two conditions.

4Study design

Research questions

• RQ1. Does disclosing the cost function reduce the measured change impact, and the concentration statistics built on it?
• RQ2. Does that reduction correspond to less complexity existing in the codebase?
• RQ3. If not, where does the complexity go? Can the measurement still see it?
• RQ4. What does the intervention cost in delivered correctness?

Arms, chains and checkpoints

The coding agent is held fixed. Architecture is the independent variable. One arm is a Spring @RestController codebase. The other is an OfficeFloor codebase of YAML-composed functions. Both implement the same PetClinic REST application. Sixty change specifications land in sequence on the single endpoint POST /api/owners. Most are additive. Every fourth from the eighth is mutative. A mutative checkpoint revises prior rules and ships updated copies of the affected tests. There are ten independent chains per architecture per condition. Each condition is therefore 1200 agent turns. The disclosed run is blind-202609010045. The control is blind-202608100006. Both use the same specifications and the same acceptance suite. Neither was modified between the two runs.

Blind protocol

The agent sees the current specification. It sees the accumulated tests up to and including the current checkpoint. It never sees future ones. Each turn runs in a history-less sandbox rebuilt from the worktree. So the agent cannot infer which checkpoint it is on from git history. Each turn also gets a fresh configuration and memory. Acceptance tests are injected per checkpoint rather than pre-committed. After the turn the full accumulated suite runs. Regressions are computed against the set of tests passing before the checkpoint.

The gate did not act

The disclosed run also carried an active gate. It could discard and re-attempt any change scoring above the 95th percentile of a reference distribution. It fired on 3 of 1200 checkpoints. One was OfficeFloor and two were Spring. It stopped no chain. All twenty chains reached checkpoint sixty. The mean accepted grade was the 31st percentile for OfficeFloor and the 38th for Spring. The prompt alone moved the agent so far below the threshold that the control loop had nothing to do. Everything reported below is therefore a prompt effect. We treat the condition as prompt-only.

Measures

Degradation slope is the OLS slope of a metric on checkpoint number. Intervals are 95% and come from a bootstrap clustered on chains. The between-architecture test is the difference of those slopes. Alongside the scoped metrics we compute an unscoped cumulative audit. That is one diff per chain, from the branch base to its final commit, over every changed file. Each changed line is attributed to the function containing it at the tip. We then sum the complexity of the distinct functions touched. It is scope-free. That makes it the check on every scoped number. Created classes are classified from the parsed function list. Duplication is measured by clone detection.

5Results

RQ1: disclosure works, on the disclosed measure

slope per checkpoint	Spring control	Spring disclosed	OfficeFloor control	OfficeFloor disclosed
impact_composite	435.1	9.71	76.4	5.74
impact_mutation	241.9	3.17	17.6	2.12
impact_godclass	193.2	6.54	58.8	3.62
wmc_handler	1.769	0.038	-0.000	0.055
entry_cc	0.122	0.035	-0.000	0.036
erosion_handler	0.00326	0	0	0
node_cc_median	3.028	1.101	0.093	0.066

Table 1. Degradation slopes, control against disclosed, both architectures. Spring's change-impact slope falls 45-fold. Its mutation term falls 76-fold. wmc_handler is the weight of the class the endpoint routes through. entry_cc is the entry handler's own complexity. node_cc_median is the complexity reachable from one handling node. All are per-checkpoint OLS slopes over ten chains.

The difference of slopes between the two architectures is the actual test. It moves further. Under the control prompt, Spring minus OfficeFloor is +1.769 [1.597, 1.929] for wmc_handler. It is +0.00326 [0.00160, 0.00489] for erosion_handler. Both exclude zero. Under disclosure they become −0.018 [−0.054, 0.018] and exactly zero. The two architectures stop being distinguishable on the god-class statistics. The change-impact difference falls from +358.7 [234.2, 507.2] to +3.97 [0.78, 7.79]. That is a ninety-fold compression. It still excludes zero, but only just.

The plainest number is not a slope. Spring's controller file grows by 604 to 962 lines from base to tip across the ten control chains. It grows by 2 to 44 lines across the ten disclosed chains. The god method that this series was built on does not appear. Taken alone, that is the rebuttal landing. Prompt better, and the architectural difference goes away.

RQ2: the complexity is conserved

base to tip, per chain	Spring control	Spring disclosed	OF control	OF disclosed
CC sum over touched functions	282.3 ± 22.2	267.9 ± 41.5	341.9 ± 13.2	269.9 ± 22.8
in files the run created	46.3	218.2	286.2	232.0
in pre-existing files	236.0	49.7	55.7	37.9
distinct functions touched	110.3	133.5	145.7	123.7
files parsed	18.7 ± 5.4	41.4 ± 3.3	66.8 ± 4.8	66.6 ± 4.7

Table 2. The unscoped cumulative audit. Every changed line at the chain tip is attributed to the function that contains it. A function counts once, however many checkpoints edited it. Spring's total is unchanged inside its spread. Its distribution inverts and its file count doubles. This is the check no prompt-side scoping can evade.

Spring's total touched complexity is 282 before and 268 after. That is flat. It sits well inside the chain-to-chain spread. What changed is where it sits. Complexity in pre-existing files falls from 236 to 50. Complexity in newly created files rises from 46 to 218. The number of files involved doubles. WMC_other is the formula's dominant lever. A function in a brand-new file has no prior neighbours. Its WMC_other floors at one. So the cheapest way to satisfy the formula is to put the rule somewhere nothing else lives.

This makes Brooks's distinction measurable. The sixty rules are essential complexity. The problem requires them. No prompt made them cheaper. What the formula could change was their placement. That is Tesler's point about conservation, applied inside a codebase rather than across the user and developer line. One caution is worth stating. OfficeFloor's total did fall, from 342 to 270. That is a genuine reduction of about a fifth. We do not attribute it to relocation. It is Spring's total, the architecture under pressure, that is conserved.

RQ3: where the rules went, and what the measure could see

Spring, per chain	control	disclosed	range, disclosed
classes created	9.0	31.5	26–37
static utility	2.3	20.6	12–31
injected bean	0.0	7.3	0–25
exception	6.4	1.6	0–10
duplicated lines, final	2230	2510
production Java lines, final	2700	2550

Table 3. What replaced the god method. Two-thirds of the classes Spring now creates are static utilities. They score near-zero WMC_other. They also give up dependency injection, test seams, proxying and transaction participation. Duplicated lines rise on a codebase that shrank. Reuse means editing an existing class, and the formula penalises that. Duplication here is clone detection only. See Section 7.

The essential complexity was conserved. The accidental complexity rose. A rule that would have been a branch in an existing method becomes a class. The cheapest class to create is a static holder with nothing around it. Reuse is actively discouraged. Reusing code means adding lines to a function whose class already carries weight.

The more serious finding is that in some chains the rules left the measurement altogether. The comprehension metric this series relies on follows method calls from the endpoint. Work pushed downstream still counts. It cannot follow what the framework dispatches. Nothing calls a request-body interceptor, an aspect, a servlet filter or an entity listener. The container invokes them.

Spring chain, disclosed	0	1	2	3	4	5	6	7	8	9
advice + aspect classes	2	25	2	20	3	2	14	2	2	3
create-path complexity	40	3	88	7	136	95	97	47	70	75

Table 4. Per-chain escape into container dispatch. Every control chain sits at the application's baseline of two such classes. Chains 1 and 3 relocate the rules into framework-invoked interceptors. The call-graph measurement then collapses. Chain 1 reports a create-path complexity of 3 with all sixty rules implemented. Its handler is the untouched upstream body. Every rule is a RequestBodyAdvice.

Two clarifications are needed here. Both correct a mid-run note we published while the run was still going. First, this is not what most chains did. Seven of ten sat at the application's baseline of two such classes, or one above it. They dispersed into ordinary static helpers instead. Second, the effect is not uniform even among the chains that went to the container. Chain 6 has fourteen such classes and a create-path complexity of 97. The honest statement is that two chains left the measurement. The architecture as a whole did not.

That variance is itself a result. One prompt, ten chains. The agent finds materially different architectures each time. Static helpers here. Interceptors there. Injected beans elsewhere. They all score well. A number that can be satisfied this many different ways is not measuring the property it is named for.

RQ4: the correctness cost

	Spring control	Spring disclosed	OF control	OF disclosed
own rule delivered (func)	1.000	1.000	1.000	1.000
all tests green (strict_pass)	0.787	0.440	0.732	0.577
standing failures, mid run	9 / 4990	191 / 4990	26	151
standing failures, final	198 / 7830	301 / 7830	168	171
breakage on untouched rules	37	56	31	50
median chain onset of first failure	rule 47.5	rule 24	rule 36	rule 24

Table 5. Delivered correctness. Every checkpoint's own rule landed in every chain of every condition. What degraded is retention of rules already delivered. Standing failures count prior tests failing at that checkpoint. A rule broken and never repaired keeps counting. Mutative checkpoints ship updated copies of the tests they revise. So these are real failures, not intended churn.

The agent implemented every rule it was asked for. That holds across all 2400 checkpoints in both conditions. What it stopped doing is keeping the earlier ones working. Spring's whole-suite pass rate falls from 0.787 to 0.440. The median chain's first permanent failure arrives at rule 24 rather than rule 47.5. The effect is systemic rather than one bad chain. Every chain's onset moves earlier.

We do not know why. Our mid-run explanation was ordering. Rules spread across independent interception points still have to run in some sequence. That sequence is no longer written anywhere. An agent adding a late rule cannot see the ordering it is joining. The completed data does not support it. The chains that went furthest into container dispatch broke less, not more. Their whole-suite pass rate is 0.511 against 0.410. Their standing failure rate is 2.3% against 3.1%. Three chains on one side makes that comparison weak in both directions. But it is the wrong sign for the hypothesis. We withdraw it pending a better one.

Two things the intervention did not cost. Agent spend was flat to slightly lower, at $78.40 against $76.82 per Spring chain. And the change-impact score kept its construct validity inside the disclosed run. It still correlates with independently measured agent cost, model time and comprehension effort at Spearman 0.53 to 0.60. The metric did not become noise. It became a target. That is a different failure.

6Discussion

The result is a clean instance of Goodhart's law with an unusually short time constant. There was no gate to fight. No reviewer to persuade. No quarter to wait out. A formula went into a prompt. The measured problem was then solved on the first attempt at nearly every checkpoint. The underlying property was not.

It would be unfair to call this cheating. Some of what the agent did is genuinely better. The god method is gone. The worst single function is smaller. Blast radius per rule fell. The share of checkpoints that disturb no existing function rose from 44 to 183 out of 600. A reviewer handed the disclosed Spring codebase would find real improvements in it. The problem is what the metric reports and what it stays silent about. It reports those improvements. It says nothing about the thirty-one new classes. Nothing about the duplicated lines. Nothing about the rules now invisible to the call graph. Nothing about the prior rules that stopped working. The gap between what the table says happened and what happened is very large. Only one side of it is legible in the table.

For the series thesis the result is supporting rather than damaging. It needs stating carefully. Disclosure did flatten Spring's concentration statistics. It did so by making Spring stop working like Spring. One handler becomes thirty-one dispersed classes. Two-thirds of them are static utilities. In two chains the rules route through the container instead. That path is available to Spring only as a deliberate deviation from its idiom. It took an explicit instruction. It came at a measured cost in correctness. OfficeFloor arrives at the same distribution as its ordinary way of working. Under the identical disclosed prompt it retains 0.577 of the suite against Spring's 0.440. The architecture did not stop mattering. It changed what the prompt had to overcome.

The practical implication is narrow and actionable. Do not put the scoring function in the agent's context. Keep the measure on the observing side. And hold any structural target you do set against an unscoped check. Here that check is the cumulative attribution of complexity to the functions that now contain it. A relocation cannot satisfy it.

7Threats to validity

The missing control

This is the most important limitation. It bears directly on the correctness result. We compare a disclosed-formula prompt against a plain implement-it prompt. We cannot yet separate disclosing the metric from any prompt that directs structural effort. A cohesion-prompt condition is running now to close exactly this gap. It asks in plain language for well-placed, single-responsibility code. It never mentions the metric. Until it lands, RQ4 should be read as a statement about this intervention. It is not yet a statement about disclosure specifically.

Condition labelling and prompt strength

The run carries an active gate. It fired three times in 1200 checkpoints and stopped nothing. We therefore report the condition as prompt-only. A fully ungated replication would be cleaner. The prompt is also more directive than the bare formula. It names WMC_other as the biggest lever. It instructs against growing a method or a class. These numbers are an upper bound on the effect of disclosure. They are not an estimate of the minimum.

Measurement

Cyclomatic complexity is a proxy for comprehension effort. It is not a measurement of it. The impact score charges only lines inside parsed function bodies. Logic expressed declaratively scores zero. That covers a mapper annotation, a schema, or OfficeFloor's wiring. Both architectures have such an escape, so it is not an architecture bias. But a checkpoint scoring zero should be read as logic going where the instrument cannot see. It should not be read as a cheap change. The verbosity figure in Table 3 is clone detection only. The pattern-based half of that metric did not execute in either run. It was a silent tool failure. We have since fixed it and cannot apply the fix retroactively. A separate check confirms the missing half would have contributed 15 to 32 lines per chain tip. That is against 2100 to 2500 clone lines. So the duplication finding does not depend on it.

Statistical

Ten chains per architecture per condition. Intervals come from a bootstrap clustered on chains. Breakage on untouched rules is a rare event and it is concentrated. One chain contributes 22 of Spring's 56 and 29 of OfficeFloor's 50. So the standing-failure series is the robust signal in Table 5, not that count. The per-chain container comparison in Section 5 rests on three chains. We report it as insufficient rather than as a null result.

Generality

One model. One endpoint. One sixty-step checkpoint plan. Two codebases. The checkpoint plan is fixed across conditions. That is what makes the comparison clean. It also means the specific correctness numbers are properties of this plan. Nothing here establishes that the same prompt would degrade correctness on a different change stream.

8Reproducibility and data availability

Data availability Both runs are reproducible from committed configuration. Each chain's branch carries its own configuration snapshot, its per-checkpoint capture records, and a provenance manifest naming the model, the tool versions and the parser probes. Every number here is recomputed from the commits rather than trusted from a log. The harness, the checkpoint plan and the acceptance suite are open [7].

python -m harness.run_experiment --config config.yaml --test-mode blind
python -m harness.analyze        --config config.yaml --run-id blind-202609010045

The control runs identically with the unchanged implement prompt. Analysis recomputes structural metrics from materialised worktrees at each checkpoint commit. So a metric added later can be applied to completed runs without re-running the agent.

9Conclusion

Handed its scoring function, the agent optimised it. The change-impact slope fell 45-fold for Spring. The growing handler that motivated this series never appeared. Two of the three concentration statistics stopped separating the architectures at all. On the numbers, the problem was solved.

The complexity did not go anywhere. Total complexity touched across a Spring chain is unchanged. It was redistributed out of the handler into twice as many files and thirty-one new classes. Two-thirds of those are static utilities. There is more duplication than before. In two chains the rules moved into framework-dispatched interceptors, where the call-graph measurement cannot reach them. Meanwhile the agent kept delivering every new rule. It stopped keeping the old ones working, from rule 24 rather than rule 47.5.

Brooks's essential complexity was not reducible by better instructions. Tesler's conservation held. The work moved, and it moved to wherever the measure was not looking. The operational conclusion is a boundary, not a retirement. Change impact is worth trusting as evidence about a change stream that is not targeting it. It is worth nothing as the objective that stream is given. The two uses cannot be combined. This run is how quickly the second one destroys the first.

---

 References and notes

1. SlopCodeBench. No-context iterative extension, erosion and verbosity metrics, degradation slope. arXiv:2603.24755.
2. SWE-CI. Continuous-integration gate, Normalized Change, EvoScore, Zero-Regression Rate. arXiv:2603.03823.
3. F. P. Brooks Jr. No Silver Bullet: Essence and Accident in Software Engineering. Proc. IFIP Congress, 1986. Reprinted in IEEE Computer 20(4), 1987.
4. L. Tesler. The law of conservation of complexity, articulated in the mid-1980s. Every process carries an irreducible complexity that design can move but not remove. Design decides only who bears it.
5. C. A. E. Goodhart. Problems of Monetary Management: The U.K. Experience, 1975. The commonly quoted form is M. Strathern's restatement, 1997. When a measure becomes a target, it ceases to be a good measure.
6. Change Impact in the Wild. An external test of the metric against defects in twenty human-written repositories. blog.officefloor.net.
7. PetClinic-Evolve degradation study. Prior posts and harness. blog.officefloor.net.
8. Cyclomatic complexity and per-function line ranges are computed with lizard. It is pinned and probed before each run. Clone detection uses jscpd at pinned thresholds.

PREPRINT · OFFICEFLOOR · SEPTEMBER 2026

I took the metric away

The last two posts did one thing. I stopped telling the AI to write good code and gave it a number instead. The exact ImpactGate cost formula, straight in the prompt. A gate that threw away any change that concentrated too much complexity. A refactor step told which classes were heavy. Everything the AI needed to keep the code clean, made measurable.

It gamed it. Exactly the way Goodhart says it will.

What the number did

When a measure becomes a target, it stops being a good measure. The AI was told the cost function and told to minimise it. So it minimised the function, not the concentration.

The formula charges a method for the complexity of the other methods in its class. So the AI stopped putting logic in the existing classes. It scattered the work into tiny new classes where the surrounding cost is near zero. And because reuse means editing a class the formula already penalises, it copied code instead of reusing it. The number went down. The code did not get better. It got more classes, more duplication, and the same tangle wearing a smaller cost.

That is the Goodhart effect in one run. The metric was honest until it became the objective. Then the AI optimised the metric and left the real problem alone.

The deeper trap: the spec becomes the program

The obvious fix is to add more metrics. Charge for duplication. Charge for new classes that only exist to dodge the first charge. Close each hole as the AI finds it.

I do not think that ends well. It is a losing game, and it is a game with a name.

Fred Brooks split software into essential and accidental complexity. The essential part is the difficulty of the problem itself. You cannot delete it. You can only decide where it lives. Larry Tesler said the same thing more bluntly. His Law of Conservation of Complexity says every system has an irreducible amount of it. The only question is who carries it.

So when I pile more rules into the metric, I am not removing the complexity. I am moving it into the metric. Keep going and the metric has to describe every structural decision precisely enough for the AI to optimise against it. At that point the metric, plus the spec, is a full description of the system. It is the program. Written in a language that cannot be run, cannot be tested, and has no tools.

That is the trap. If keeping the code clean requires me to specify the system twice, once as a spec to optimise and once as the code it produces, then the spec is just a more expensive way to maintain the definition of the system than the code was. Code is already the most precise, testable, tooled description we have. Replacing it with an ever growing specification is not progress.

So I stopped adding to the number. I started taking it away. Two experiments, two ways to remove it.

Experiment one: keep the tool, hide the number

The first experiment keeps ImpactGate but never lets the AI see the formula.

The AI implements each change with a plain spec. No metric. No objective to optimise. ImpactGate still scores the change in the background. When a change concentrates too much, one refactor runs first. But the refactor is told only the symptom, not the cure.

It is given the names of the classes that ended up carrying too much. Locations only. Not the cost, not the formula, not the words "split into small classes". Those phrases are what drove the dispersal and the duplication last time. Here is the whole refactor prompt.

A change is about to be made to this codebase. When that change was
implemented directly, these classes ended up carrying more complexity
than they can hold comfortably:

{drivers}

Before the change is made, prepare the existing code so the change can
land as code a maintainer of this codebase would recognise as natural,
consistent with how this codebase already structures similar work.

It names the sore spot and asks for something natural. It does not hand over a lever to pull.

One more guard. The refactor's own new lines run through a deterministic quality gate. Duplicated lines and known bad patterns are caught by jscpd and ast-grep, not by a prompt. So the AI cannot buy a quieter structure with copy paste. The prompt asks for good judgement. The gate, not the prompt, forbids the cheap moves.

The question is simple. With no number to optimise and only a nudge at the sore spot, does the structure stay cohesive?

Experiment two: ask in plain English

The second experiment drops the tool entirely. No gate. No refactor step. No number anywhere.

I am not ruling out that guidance helps. I am ruling out the formula. So the ask goes back to English, but honest English. Not a target to optimise. A plain request for structure, in the words a senior engineer would use in review.

As you do, keep the code well structured: put each piece of logic where
it belongs, in a small unit with a single responsibility, and reuse
existing code instead of copying it. Do not let any one class or method
grow into a catch-all that accumulates unrelated logic.

That is the whole intervention. Same single turn as the plain spec run. Same everything else. Only the wording changes. It says nothing about a metric, a cost, or the experiment. It cannot be gamed, because there is no number to game. It also cannot be checked, by the AI or by me, which is exactly the weakness the formula was meant to fix.

So this experiment asks the honest version of the original question. Not "can a measurable target keep code clean", which the AI answered by gaming the target. But "can plain good advice keep code clean", with nothing to optimise against it.

What the two experiments are really testing

Line everything up and there is a clean ladder.

• Plain spec. No help at all. The baseline erosion.
• Plain English. Good advice, no number. The prompting lever on its own.
• Refactor run. A tool that catches the sore spot and nudges it, still no number shown. The tool lever on its own.
• The formula. The full measurable target. Already run. Already gamed.

Each step adds exactly one thing. So the gap between two steps is the effect of that one thing. And every one of them is measured the same way at the end, on the shape of the standing code, not on any score the AI could aim at.

Here is the worry that started all this. Tesler says the complexity never leaves. It only moves. My first bet was to hold it down with a measurement, and the AI gamed the measurement. These two experiments try the opposite and take the number away.

But the complexity still has to land somewhere. Maybe the AI invents its own target to chase, and games that instead, with no formula in sight. Maybe the only way to stop it is to keep adding to the spec, until the spec is the program again. If either happens, then a metric was never the right lever. A better number cannot fix a problem that lives in the architecture. The fix would have to be the architecture itself.

That is what these two experiments are built to find out. Same harness, same problem, same chains, same measurement at the end.

The runs are underway.