We told the AI about the Change Impact

A series on how software architecture shapes AI-driven code degradation. This is the plain version of a result. The paper has the numbers and the caveats.

We have been running an experiment. An AI agent gets sixty change requests, one after another, all landing on the same REST endpoint. Add a validation rule. Change how phone numbers are stored. Add a duplicate check. Sixty of them, in order, with the full test suite run after each one.

We do this twice. Once on a normal Spring codebase. Once on the same application built with OfficeFloor. Then we look at what sixty changes did to each one.

The Spring result has been the same every time. The create-owner handler method grows. Rule after rule lands in it. By the end it is the biggest thing in the codebase and every new rule means opening the same enormous method again.

And there has always been an obvious objection to that. Nobody told the AI not to do it.

So we told it

We have a metric called change impact. It scores a change by how much complexity it disturbs, rather than by how many lines it edits. Touching a small method in a small class is cheap. Adding ten lines to a huge method in a huge class is expensive.

Roughly, for every method you change:

cost = (how heavy the class already is)
     x (how complicated the method is)
     x (how many of its lines you changed)

Add that up for every method you touched. Multiply by the number of files. Lower is better.

This time we put that formula in the prompt. Every single change request. We told the AI exactly how it was being scored, and we told it which part of the formula mattered most.

It worked immediately

We had also built a safety net. If a change scored badly, we would throw it away and make the AI refactor first. We expected that to be doing the work.

It fired three times. Out of twelve hundred changes.

The prompt alone was enough. Told what the score was, the AI wrote code that scored well on it, first try, nearly every time.

And the god method never appeared. Here is how much the controller file grew over sixty rules:

Spring controller, lines added over 60 rules	Ten runs
Normal prompt	604 to 962
Told the formula	2 to 44

That is the objection landing. Prompt better and the problem goes away. If we stopped here, this post would say architecture does not matter much and good instructions do.

Then we looked at where the code went

We have a second check that does not care about any of our metrics. It takes the finished codebase, finds every line the run changed, and works out which method that line ended up in. Then it adds up how complicated all those methods are. It is deliberately dumb. It just asks how much logic now exists and where it lives.

Spring, after 60 rules	Normal prompt	Told the formula
Total logic written	282	268
...sitting in brand new files	46	218
...sitting in files that already existed	236	50
Number of files involved	19	41

Look at the first row. The amount of logic is the same. 282 before, 268 after. Nothing got simpler.

Now look at the next two rows. It all moved. Out of the files that existed, into files the AI created. Twice as many files.

This is not the AI being clever or sneaky. Look at the formula again. The first term is how heavy the class already is. A brand new file has nothing in it. So that term is as small as it can possibly get. If you want a low score, the cheapest thing you can do is put your code somewhere nothing else lives.

So it did. Sixty times.

What thirty-one new classes look like

Per run, the normal prompt created about nine new classes. Told the formula, it created about thirty-one. Roughly twenty of those were static utility classes. Classes with one static method, holding one rule, and nothing else.

On the score, that is perfect. A static method in an otherwise empty class has almost no surrounding complexity to pay for.

In a real Spring codebase, it costs you things a junior engineer runs into fast. A static method is not a bean. You cannot inject anything into it. You cannot swap it out in a test. Spring cannot wrap it in a transaction or a proxy. You have made the metric happy and given up most of what the framework is for.

The duplication went up too, and for a reason worth understanding. Reusing an existing helper means adding a line to a method in a class that already has weight. The formula charges you for that. Writing your own copy in a fresh file is free. So the AI wrote its own copy. Duplicated lines went from about 2230 to about 2510, in a codebase that had got smaller overall.

One run vanished completely

We ran ten independent Spring chains. Most of them dispersed into helper classes as described. Two did something else.

In one of them, the create-owner handler at the end of sixty rules is the method we started with. Map the request. Save the owner. Set the location header. Return 201. That is it.

Every one of the sixty rules is a @RestControllerAdvice class. Eighteen of them. They run before the handler is ever called, because Spring invokes them, not the handler.

Our comprehension metric follows method calls from the endpoint. It is a good metric. It was added because a reader correctly pointed out that a pipeline can hide work in its later stages, and following the calls fixes that.

It cannot follow something nothing calls. An interceptor is invoked by the framework. So for that run, our metric reports the create path as having a complexity of 3, for a codebase implementing sixty business rules.

That is not a clean codebase. If you are asked to change the phone number rule, you still have to find it first, and finding it just got much harder. The number got quieter. The code did not get simpler.

The part that actually matters

All of the above is arguing about metrics. This part is not.

Spring	Normal prompt	Told the formula
Implemented the rule it was asked for	every time	every time
Whole test suite still green	79% of changes	44% of changes
First rule permanently broken at	rule 47	rule 24

The AI still did the job in front of it. Every time. What it stopped doing was keeping the previous fifty-nine rules working.

Something broke, earlier and more often, and none of the structural metrics showed it. The tests showed it.

We had a theory. Rules scattered across separate interceptors still have to run in some order, that order is no longer written down anywhere, and an AI adding rule 40 cannot see the ordering it is joining. It is a nice theory. The finished data does not support it. The runs that leaned hardest on interceptors actually broke slightly less. So we do not know why yet, and we are saying so rather than keeping a tidy explanation that the numbers disagree with.

This is a very old idea

Two of them, actually.

Fred Brooks, in 1986, split the difficulty of software into two parts. Essential complexity is the problem itself. Sixty business rules are sixty business rules. Accidental complexity is the mess we add on top through how we choose to build it. His argument was that no tool removes the essential part.

That is exactly the first row of our second table. 282 before, 268 after. The rules are the rules. No prompt made them cheaper.

Larry Tesler put it a different way in the 1980s, usually called the law of conservation of complexity. Complexity does not disappear. It moves. Design decides who has to deal with it, not whether anybody does.

That is the rest of the table. The complexity moved out of the handler and into forty-one files, and in two runs it moved somewhere our tooling could not follow at all.

And the reason it moved is Goodhart's law, in its usual form: when a measure becomes a target, it stops being a good measure. We knew that. We still did not expect it to happen this completely, in one run, from one paragraph of prompt, with no gate ever firing.

What to take from this

If you are early in your career and working with AI tools, this is the practical version.

A green metric is not the same as good code. When a number improves a lot and quickly, ask what moved. Not what got deleted. Things rarely get deleted.

Be suspicious of a class that exists to hold one rule and nothing else. It is often a real improvement. It is also the cheapest way to make almost any code metric look better, so it is worth checking which one you are looking at.

Static helpers look free and are not. You give up injection, mocking, transactions and proxying. If someone, human or AI, is producing a lot of them quickly, that is worth a conversation.

If you cannot find where a rule runs, the codebase got harder, whatever the dashboard says. Being able to open one method and read what happens is worth a lot.

And keep your tests. In this experiment every structural metric we own said the code got better. The acceptance suite was the only thing that noticed rules quietly breaking from change 24 onwards. That is not a small detail. It is the whole reason we caught this.

What we are not claiming

We told the AI to optimise a formula and things got worse. We have not yet shown that a normal request for good structure would do the same. There is a difference between "here is the arithmetic, minimise it" and "please keep this code well organised", and we are running that second version now.

Until that finishes, the honest claim is narrow. Do not hand an AI the metric you are judging it by. Use the metric to watch. Do not use it as the goal. Those are two different jobs and it can only do one of them.