Part of a series on how software architecture shapes AI driven code degradation. This post explains a group of measurements on their own. Each metric gets its definition, its figure, and its numbers.
Structural impact: the composite score
impact_composite · ↓ lower is better · defined in this harness (harness/metrics.py)
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
One number for how much a change cost the structure. It is blast radius weighted by the complexity of the context that was disturbed. Editing a method inside a heavy god class costs far more here than the same edit inside an isolated unit, which is the whole design.
impact_composite = files_changed · Σf ∈ changed cost(f)
Where. cost(f) is the per function term. WMC_other(f) is the summed cyclomatic complexity of the other methods in f's class, which stands for the context you must hold in your head to change f safely. A brand-new class has WMC_other = 0, which the max(…, 1) floors to 1, so a new isolated unit still costs something. That floor is what closes the fragmentation loophole. Δlines(f) is the changed-line count in f, and for a new function it is that function's own size. files_changed is the number of distinct production Java files the commit touched, applied as a multiplier, so scattering one rule across many classes is not free. The sum runs over every changed function, both modified and new. A within-commit rename is charged as a mutation rather than a free addition when the two bodies have a line-set Jaccard similarity of at least 0.6.
In this harness. metrics.impact_stats parses each touched file at both the previous and the current commit with lizard, matches functions by name, and falls back to body similarity for renames. Note what it cannot see: only lines inside a parsed function body count, so logic expressed declaratively, in a MapStruct expression, in openapi.yml, in schema.sql or in OfficeFloor's wiring, scores zero. Both arms have that escape hatch, so it is not an arm bias, but read a zero as “the logic went where this metric cannot look” rather than as “the change was cheap”.
How to read it. The scale is large and heavily skewed, because it is a product of four terms. The shape of the line matters far more than its value.
Structural impact: disturbing existing functions
impact_mutation · ↓ lower is better · this harness
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
The half of the impact score that comes from modifying code that already existed. This is the component that carries the discrimination between the two architectures, because the context weight means a mandated rule revision is genuine architectural signal rather than spurious re-touching.
Where. cost(f) is the per function term. WMC_other(f) is the summed cyclomatic complexity of the other methods in f's class, which stands for the context you must hold in your head to change f safely. A brand-new class has WMC_other = 0, which the max(…, 1) floors to 1, so a new isolated unit still costs something. That floor is what closes the fragmentation loophole. Δlines(f) is the changed-line count in f, and for a new function it is that function's own size. files_changed is the number of distinct production Java files the commit touched, applied as a multiplier, so scattering one rule across many classes is not free. The sum is restricted to functions that existed at the previous commit and were modified, plus those detected as renames by the 0.6 Jaccard rule.
In this harness. Same single pass as the composite.
How to read it. This is where the two architectures separate most sharply. If the composite moves and this does not, the movement was all in additions.
Structural impact: adding new functions
impact_godclass · ↓ lower is better · this harness
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
The half of the impact score that comes from new code: new files and new methods added to existing classes. It is reported so the composite's behaviour can be attributed to the right half.
Where. cost(f) is the per function term. WMC_other(f) is the summed cyclomatic complexity of the other methods in f's class, which stands for the context you must hold in your head to change f safely. A brand-new class has WMC_other = 0, which the max(…, 1) floors to 1, so a new isolated unit still costs something. That floor is what closes the fragmentation loophole. Δlines(f) is the changed-line count in f, and for a new function it is that function's own size. files_changed is the number of distinct production Java files the commit touched, applied as a multiplier, so scattering one rule across many classes is not free. The sum covers functions with no counterpart at the previous commit. For these Δlines(f) is the new function's own line count, and WMC_other is 0 in a brand-new class, floored to 1.
In this harness. Same single pass as the composite.
How to read it. Because of the floor and the spread multiplier, both architectures pay something for additions. So this component does not separate them cleanly, and that is correct. The discrimination is supposed to live in the mutation term.
Structural impact (composite): purely additive rules only
impact_composite_add · ↓ lower is better · this harness, filtered view
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
The same impact score, restricted to the change requests that only add a rule and never revise an earlier one. This is the easy case, where an architecture is not asked to change its mind about anything.
Where. A change request is additive when the checkpoint plan declares no mutates list for it. The filtered field is left blank, not zero, on the other checkpoints. Blank matters: a zero would drag the mean of this view toward nothing on every revision checkpoint, which would make the two views incomparable.
In this harness. Derived in the gallery and in analyze from the base field and the checkpoint_type column. It is never written to the per-checkpoint CSV.
How to read it. If a condition only looks good here, it only looks good when nothing has to change.
Structural impact (composite): rule-revision rules only
impact_composite_mut · ↓ lower is better · this harness, filtered view
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
The same impact score, restricted to the change requests that deliberately revise an earlier rule. Fourteen of the sixty change requests are of this kind, and they are placed deliberately rather than at random.
Where. A change request is mutative when the checkpoint plan declares a mutates list, naming the earlier rules it is allowed to change. Additive checkpoints are blanked, for the same reason as above.
In this harness. Same derivation. Unlike the correctness metrics, the impact score keeps the mutative checkpoints in its base field rather than discounting them, because the context weight makes a mandated revision genuine architectural signal.
How to read it. The interesting case. Revising an existing rule is where an architecture either pays for having isolated the concern or does not. In the control run the concentrated arm paid roughly 36,000 per revision. The distributed arm paid 4,800.
Structural impact (mutation of existing functions): purely additive rules only
impact_mutation_add · ↓ lower is better · this harness, filtered view
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
The same impact score, restricted to the change requests that only add a rule and never revise an earlier one. This is the easy case, where an architecture is not asked to change its mind about anything.
Where. A change request is additive when the checkpoint plan declares no mutates list for it. The filtered field is left blank, not zero, on the other checkpoints. Blank matters: a zero would drag the mean of this view toward nothing on every revision checkpoint, which would make the two views incomparable.
In this harness. Derived in the gallery and in analyze from the base field and the checkpoint_type column. It is never written to the per-checkpoint CSV.
How to read it. If a condition only looks good here, it only looks good when nothing has to change.
Structural impact (mutation of existing functions): rule-revision rules only
impact_mutation_mut · ↓ lower is better · this harness, filtered view
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
The same impact score, restricted to the change requests that deliberately revise an earlier rule. Fourteen of the sixty change requests are of this kind, and they are placed deliberately rather than at random.
Where. A change request is mutative when the checkpoint plan declares a mutates list, naming the earlier rules it is allowed to change. Additive checkpoints are blanked, for the same reason as above.
In this harness. Same derivation. Unlike the correctness metrics, the impact score keeps the mutative checkpoints in its base field rather than discounting them, because the context weight makes a mandated revision genuine architectural signal.
How to read it. The interesting case. Revising an existing rule is where an architecture either pays for having isolated the concern or does not. In the control run the concentrated arm paid roughly 36,000 per revision. The distributed arm paid 4,800.
Structural impact (new-function additions): purely additive rules only
impact_godclass_add · ↓ lower is better · this harness, filtered view
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
The same impact score, restricted to the change requests that only add a rule and never revise an earlier one. This is the easy case, where an architecture is not asked to change its mind about anything.
Where. A change request is additive when the checkpoint plan declares no mutates list for it. The filtered field is left blank, not zero, on the other checkpoints. Blank matters: a zero would drag the mean of this view toward nothing on every revision checkpoint, which would make the two views incomparable.
In this harness. Derived in the gallery and in analyze from the base field and the checkpoint_type column. It is never written to the per-checkpoint CSV.
How to read it. If a condition only looks good here, it only looks good when nothing has to change.
Structural impact (new-function additions): rule-revision rules only
impact_godclass_mut · ↓ lower is better · this harness, filtered view
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
The same impact score, restricted to the change requests that deliberately revise an earlier rule. Fourteen of the sixty change requests are of this kind, and they are placed deliberately rather than at random.
Where. A change request is mutative when the checkpoint plan declares a mutates list, naming the earlier rules it is allowed to change. Additive checkpoints are blanked, for the same reason as above.
In this harness. Same derivation. Unlike the correctness metrics, the impact score keeps the mutative checkpoints in its base field rather than discounting them, because the context weight makes a mandated revision genuine architectural signal.
How to read it. The interesting case. Revising an existing rule is where an architecture either pays for having isolated the concern or does not. In the control run the concentrated arm paid roughly 36,000 per revision. The distributed arm paid 4,800.
No comments:
Post a Comment