Part of a series on how software architecture shapes AI driven code degradation. This post explains a group of measurements on their own. Each metric gets its definition, its figure, and its numbers.
Complexity inequality across files (Gini)
ccdist_file_gini · ↓ lower is better · Gini 1912, applied to per-file cyclomatic complexity
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
Take every file's total complexity and ask how unequally it is distributed. It is the same statistic economists use for income. 0 means every file carries the same complexity. 1 means one file carries it all. This is the headline concentration metric of the whole experiment, because it is the only one of the four that measures shape alone.
Where. The input is a vector of per-unit weights x1 … xn, one per unit, zeros dropped. pi = xi / Σx is unit i's share of the total. For the Gini the weights must be sorted ascending, so i is the rank from lightest to heaviest and the leading factor runs from negative to positive. xi is file i's summed CC over the functions in it. The result is undefined, and blank, for fewer than two files.
In this harness. placement.gini over per-file CC totals at the checkpoint commit. Files with zero complexity are dropped before sorting, because an interface or a constants holder would otherwise read as poverty.
How to read it. This is the one to lead with. Gini is scale-free: it describes the shape of the distribution and is insensitive to how many units it is spread over. An architecture cannot improve its Gini by splitting files into more files of the same shape. So if Gini separates the arms, the distribution really is more unequal.
Complexity concentration across files (HHI)
ccdist_file_hhi · ↓ lower is better · Herfindahl-Hirschman index (Hirschman 1945), over per-file CC shares
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
The sum of squared shares. It is the standard concentration index from competition regulation, where it decides whether a market is too concentrated. One file holding everything scores 1. A hundred equal files score 0.01. Squaring is what makes it responsive: it is dominated by the largest holder, so it moves sharply when one file runs away.
Where. The input is a vector of per-unit weights x1 … xn, one per unit, zeros dropped. pi = xi / Σx is unit i's share of the total. The index ranges from 1/n, for perfectly even, up to 1, for everything in one unit. That lower bound is the whole problem with it here, because it depends on n.
In this harness. placement.hhi over per-file CC totals.
How to read it. Sharper than the Gini when one file is running away, because squaring weights the leader.
Share of all complexity in the single heaviest file
ccdist_file_top1 · ↓ lower is better · concentration ratio CR1
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
What fraction of the entire application's complexity lives in its one biggest file. This is the most directly interpretable number on the page. It reads as a sentence: one file in this codebase holds this much of all its decisions.
Where. The input is a vector of per-unit weights x1 … xn, one per unit, zeros dropped. pi = xi / Σx is unit i's share of the total. No sorting or squaring. It is simply the largest single share.
In this harness. placement.top_share with k=1, over per-file CC totals.
How to read it. If this climbs toward a third as rules accumulate, the god-file story is not a metaphor. It is also the easiest figure to quote to someone who does not want a statistics lesson.
Share of all complexity in the heaviest five files
ccdist_file_top5 · ↓ lower is better · concentration ratio CR5
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
The same question widened to five files, so a codebase cannot look healthy simply by splitting its god file in two.
Where. The input is a vector of per-unit weights x1 … xn, one per unit, zeros dropped. pi = xi / Σx is unit i's share of the total. x(i) denotes the weights sorted descending, so the sum is over the five heaviest. With fewer than five non-zero files the value is 1 by construction.
In this harness. placement.top_share with k=5.
How to read it. Resistant to the cosmetic split. If the top-1 share falls but this does not, the complexity was moved next door rather than distributed.
Complexity spread across files (normalised entropy)
ccdist_file_hnorm · ↑ higher is better · Shannon entropy, normalised by log n
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
How evenly complexity is spread, on a scale of 0 to 1 where 1 is perfectly even. It is the information-theoretic mirror of the concentration indices, and it reads as the number of bits you would need to say which file a randomly chosen unit of complexity came from.
Hnorm = H / ln n
Where. The input is a vector of per-unit weights x1 … xn, one per unit, zeros dropped. pi = xi / Σx is unit i's share of the total. n is the number of non-zero units. Dividing by ln n is the maximum entropy that many units can have, which is what puts the result on a 0 to 1 scale and removes most of the unit count advantage that HHI gives away. The base of the logarithm cancels in the ratio, so natural log here and log base 2 elsewhere give the same normalised number.
In this harness. placement.norm_entropy over per-file CC totals. Undefined, and blank, for fewer than two files.
How to read it. Note the direction flips. Higher means more distributed. Because it is normalised it makes a useful referee between the Gini and the HHI when those two disagree.
Complexity inequality across functions (Gini)
ccdist_fn_gini · ↓ lower is better · Gini 1912, applied to per-function CC
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
The same inequality question one level down, across individual functions rather than files. Files can be split for cosmetic reasons. A function is the unit a developer actually reads at one sitting, so this is the god-method question rather than the god-file one.
Where. Same formula as the file Gini, with xi now the cyclomatic complexity of one function, sorted ascending. n is the number of functions with non-zero complexity.
In this harness. placement.gini over the per-function CC column.
How to read it. This asks whether one method is becoming the place decisions go. It is harder to game than the file version, because you cannot split a method without changing the code.
Complexity concentration across functions (HHI)
ccdist_fn_hhi · ↓ lower is better · Herfindahl-Hirschman index over per-function CC shares
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
Concentration of decisions into single methods. The god-method signal, as opposed to the god-class signal.
Where. Same formula as the file HHI, with pi now one function's share of total cyclomatic complexity.
In this harness. placement.hhi over the per-function CC column.
How to read it. Squaring means one very heavy method dominates the figure, which is the behaviour you want from a god-method detector.
Complexity concentration across packages (HHI)
ccdist_pkg_hhi · ↓ lower is better · Herfindahl-Hirschman index over per-package CC shares
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
Concentration at the coarsest level, across whole packages. It survives any amount of file-level reshuffling inside a package.
Where. pi is package i's share of total cyclomatic complexity, where a package is the directory path below the source root. n is the package count, which is small, so this index sits much higher than the file or function versions and must not be compared against them numerically.
In this harness. placement.hhi over CC grouped by placement._package_of.
How to read it. If a condition improves the file numbers but not this one, the rules were split into more files inside the same package. That is a smaller change than it looks.
Class-weight concentration (HHI over WMC)
wmcdist_class_hhi · ↓ lower is better · Herfindahl-Hirschman index over per-class WMC, via CK
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
The concentration question over classes, weighted by how much complexity each class's methods carry. It is the class-level view of the god-object question, computed by the independent parser.
Where. WMC(C) is the sum of cyclomatic complexity over the methods of class C, as CK reports it. Unlike the file-based indices this one sees nested and inner classes as their own units, because CK resolves real class declarations rather than assuming one class per file.
In this harness. placement.hhi over CK's per-class WMC column.
How to read it. A useful cross-check on the file-level indices, since it uses a different notion of a unit and a different parser.
Cognitive-complexity concentration across functions
cogdist_fn_hhi · ↓ lower is better · HHI over per-function cognitive complexity (PMD)
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
Concentration measured in units of human difficulty rather than branch count. It asks whether the nesting is pooling in one place, which is a different question from whether the branches are.
Where. cognitive(f) is the Campbell measure defined in the amount group: one point per flow-breaking structure plus one per level of nesting it sits inside.
In this harness. placement.hhi over PMD's per-method cognitive complexity.
How to read it. If concentration shows up in cyclomatic terms but not cognitive terms, the concentrated method is long but flat. If it shows up in both, it is long and deeply nested. That is the genuinely hard kind.
Halstead-volume concentration across files
voldist_file_hhi · ↓ lower is better · HHI over per-file Halstead volume
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
Concentration measured by vocabulary rather than by control flow. It is the control-flow-free confirmation that the concentration result is not an artefact of how branches are counted.
Where. V(file) is that file's Halstead volume, as defined in the amount group. Because literals are collapsed to one placeholder operand, this cannot be moved by editing message text.
In this harness. placement.hhi over the harness's per-file Halstead volumes.
How to read it. A branch-free second opinion on the file-level concentration result.
Heaviest class in the codebase (WMC)
wmc_max · ↓ lower is better · Chidamber & Kemerer 1994, via lizard
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
Weighted methods per class is the sum of the cyclomatic complexities of every method in a class. The heaviest such class is the god-class candidate. It catches what a per-method erosion threshold cannot: a controller that stays tidy method by method, while accumulating twenty rules' worth of methods, becomes a god class without any single method ever crossing CC 10.
wmc_max = maxC ∈ touched WMC(C)
Where. C ranges over the touched subsystem, which is the production Java files changed since the baseline commit. That scoping is deliberate: it tracks what the agent is building rather than what shipped with the framework. On the lizard side a class is identified with a file, on the assumption of one top-level class per Java file.
In this harness. metrics.wmc_stats groups lizard's functions by file, sums CC per group and returns the worst, along with its name, method count and line count.
How to read it. Rising means one class is accumulating everything.
Weight of the handler class (like-for-like)
wmc_handler · ↓ lower is better · C&K WMC, pinned to the class the endpoint routes through
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
The same god-class number, but measured on the class that actually handles the create-owner request, in both architectures. Pinning it to a role rather than to whichever class happens to be heaviest is what makes it a fair comparison.
Where. H is the handler class, which is the file containing the function the create endpoint routes through. The harness finds it per arm from the configured entry_handler pattern, so it resolves to the Spring controller in one arm and to the wired create function's class in the other. The field is blank until that class exists.
In this harness. Same scoping as handler erosion, in metrics.entry_handler_stats and its WMC companion.
How to read it. A rising line means the front door is turning into a god class. This is the like-for-like god-class comparison, same role, both arms.
Weight of the handler class (independent parser)
ck_handler_wmc · ↓ lower is better · CK tool (Aniche 2015), Eclipse JDT parser
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
The handler-class weight again, from the second tool. A cross-check that neither parser is silently failing on the handler file, which is the one file on which the whole concentration argument rests.
Where. H is matched by class name rather than by file path here, because CK reports per class. Differences from the lizard figure are usually nested classes, which CK attributes separately.
In this harness. Read from CK's per-class CSV, filtered to the handler class name.
How to read it. Agreement with the lizard figure is the boring result you want.
Complexity of the front-door method
entry_cc · ↓ lower is better · McCabe 1976, scoped to the entry function
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
The cyclomatic complexity of the single method the HTTP request lands in. That is the method a developer opens first when asked to change this endpoint, so it is the purest available measure of whether the front door bloats.
Where. e is the one function the create endpoint routes through, which is addOwner in the Spring arm and the wired create function's service method in the OfficeFloor arm. CC(f) is the cyclomatic complexity of function f, as reported by lizard: one plus the number of decision points, counting if, for, while, case, catch, the ternary operator, and each && or ||.
In this harness. metrics.entry_handler_stats matches the configured entry pattern against lizard's function list and reports that function's CC, line count and name.
How to read it. The purest front-door measure, and the easiest to explain.
Worst single function in the touched subsystem
hotspot_cc · ↓ lower is better · lizard
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
The highest cyclomatic complexity of any one function among the files this run has changed. Scoping it to changed files means it tracks what the agent is actually building, rather than whatever the framework shipped with.
Where. touched is the set of production Java functions in files changed since the baseline commit. The harness also records which function it was, as hotspot_fn, so the figure can be traced to a name.
In this harness. metrics.hotspot_stats over the dynamically scoped subsystem.
How to read it. A climbing line means the worst thing the agent has written keeps getting worse. Because it is a maximum it is noisy, so read the trend.
Structural erosion, whole application
erosion · ↓ lower is better · SlopCodeBench (arXiv:2603.24755) Eq. 3
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
What fraction of the codebase's complexity sits inside functions that are over the complexity threshold. Each function is given a mass that combines its branching with its size, and erosion is the share of that mass held by functions judged too complex. It is the benchmark's own headline measure, which is why it is reported here at all.
erosion = Σf : CC(f) > 10 mass(f) / Σf ∈ F mass(f)
Where. CC(f) is the cyclomatic complexity of function f, as reported by lizard: one plus the number of decision points, counting if, for, while, case, catch, the ternary operator, and each && or ||. SLOC(f) is the function's non-comment line count. The square root is the benchmark's choice: it means size matters but with diminishing returns, so a 400-line function is not simply scored as ten times a 40-line one. The threshold of CC > 10 is McCabe's own recommended limit, not a value chosen here. The population is production Java only, meaning the files matched by the arm's source_globs at that checkpoint's commit. Tests are excluded. OfficeFloor's YAML wiring is counted in a separate pool and is never folded into a Java denominator, because mixing them would let a wiring-based architecture dilute any per-line metric.
In this harness. metrics.erosion_detail. The harness records the numerator and denominator separately, as erosion_high_mass and erosion_total_mass, so the ratio is reproducible from the CSV without rerunning anything.
How to read it. Kept for comparability with the published benchmark.
Structural erosion, changed files only
erosion_scoped · ↓ lower is better · SlopCodeBench Eq. 3, scoped to the touched subsystem
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
The same erosion ratio, restricted to files this run has actually modified, so untouched framework code cannot dilute it.
Where. Same mass and same threshold as the whole-app version. touched is the production Java functions in files changed since the baseline commit, which grows as the run proceeds. The harness records the subsystem's function count as subsystem_nfns so the denominator's size is visible.
In this harness. metrics.erosion_detail over the dynamically scoped subsystem.
How to read it. A tighter version of the whole-app number.
Structural erosion of the handler class
erosion_handler · ↓ lower is better · SlopCodeBench Eq. 3, scoped to the entry handler's own class
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
Erosion measured only inside the class that handles the request. Scoping it this way excludes the leaf algorithms that swamp the two wider versions. What is left is the clean concentration signal: does the endpoint's own handler surface rot as rules accumulate?
Where. H is the functions of the handler class only, found by the same entry pattern as the handler WMC. The harness records erosion_handler_class and erosion_handler_nfns so you can confirm which class was measured and how many functions were in the denominator.
In this harness. metrics.handler_scoped_erosion.
How to read it. This is the erosion number that tests the thesis.
God classes detected
pmd_god_classes · ↓ lower is better · Lanza & Marinescu 2006 thresholds, via PMD's GodClass rule
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
How many classes trip a published god-class detector. The value of this metric is that nobody in this experiment chose the thresholds. It is an externally defined, binary verdict, which makes it immune to the accusation that the scoring was tuned to the result.
pmd_god_classes = | { C : GodClass(C) } |
Where. All three conditions must hold. WMC is weighted methods per class, as above. ATFD is Access To Foreign Data: the number of distinct fields of other classes this one reaches into, which is what distinguishes a class doing too much from a class that is merely large. TCC is tight class cohesion, defined in the cohesion group, so the third condition says the class is also internally incoherent. The constants 47, 5 and 1/3 are Lanza and Marinescu's.
In this harness. PMD's GodClass rule, from the same single PMD spawn as the complexity measures. The harness counts distinct violating classes.
How to read it. A step change in this line is a strong, externally validated statement that the codebase acquired a god class at that change request.
Is the handler class itself a god class
pmd_handler_is_god_class · ↓ lower is better · PMD GodClass rule, evaluated on the handler class
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
Whether the class handling the endpoint trips the published detector. It is binary, externally defined, and about the specific class both architectures agree is the entry point, which makes it the single most quotable structural result on this page.
plotted value = (1/K) Σk=1..K valuek(c)
Where. H is the handler class. GodClass is the three-condition test above. K is 10, the number of runs, so the plotted line is the fraction of runs in which the front door has become a god class by that change request.
In this harness. The harness passes the handler class name to placement.pmd_metrics_from_report and records whether PMD flagged it.
How to read it. Read the line as: in what fraction of runs has the front door become a god class by change request N.
Data classes detected
pmd_data_classes · · descriptive · PMD DataClass rule
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
Classes that are mostly fields and accessors with little behaviour. This is context rather than a finding. It is how you tell whether a heaviest-class result is about a controller full of decisions or an entity full of getters.
Where. PMD's DataClass rule fires on a class with a high proportion of public accessors, few real methods and low complexity per method. It is the complement of the god class: too little behaviour rather than too much.
In this harness. Same single PMD spawn, counting distinct violating classes.
How to read it. Use it to interpret the role-blind heaviest-class metric. A rising data class count in one arm usually means its entity is growing accessors.
No comments:
Post a Comment