Part of a series on how software architecture shapes AI driven code degradation. This post explains a group of measurements on their own. Each metric gets its definition, its figure, and its numbers.
Total cyclomatic complexity
total_cc · = prediction: no arm difference · McCabe 1976, via lizard
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
The total number of independent paths through the whole application. Every branch point in every function, summed. It is the standard answer to how much decision-making a codebase contains, and it is the primary test of Tesler's conservation law here: the sixty change requests demand a certain number of decisions, and no architecture can delete them.
Where. F is every function in production Java at that checkpoint's commit. CC(f) is the cyclomatic complexity of function f, as reported by lizard: one plus the number of decision points, counting if, for, while, case, catch, the ternary operator, and each && or ||. The population is production Java only, meaning the files matched by the arm's source_globs at that checkpoint's commit. Tests are excluded. OfficeFloor's YAML wiring is counted in a separate pool and is never folded into a Java denominator, because mixing them would let a wiring-based architecture dilute any per-line metric.
In this harness. lizard parses every matched file and reports one record per function. The harness sums the cyclomatic complexity column. A parser that fails on a file yields no functions for it, which would make that file look free, so parser_selftest must pass before a run or an analysis is trusted.
How to read it. The prediction is that the two lines sit on top of each other. If a condition lowers this, it either skipped work or pushed logic somewhere the parser cannot read. Check the validity group before celebrating.
Total cognitive complexity
pmd_cognitive_total · = prediction: no arm difference · Campbell / SonarSource 2018, via PMD
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
A complexity measure built around how hard code is for a human to follow, rather than how many paths it has. Nesting is penalised heavily. A long flat sequence of independent checks is forgiven. That makes it the one measure in this group that could separate a deeply nested god method from a long flat dispatch, which is exactly the distinction cyclomatic complexity is blind to.
pmd_cognitive_total = Σf ∈ F cognitive(f)
Where. structures(f) are the flow-breaking constructs in f: if, else if, else, switch, each loop, each catch, each ternary, and each run of mixed && or || operators. nesting(s) is how many enclosing structures s sits inside. Some constructs take the increment without contributing nesting, which is what forgives a flat sequence: ten sibling if statements score 10, while ten nested ones score 55.
In this harness. PMD computes it with its own Java parser. The harness spawns PMD once per checkpoint for both of its rulesets and splits the merged report, because a JVM launch per ruleset would dominate the analysis time.
How to read it. Agreement with total cyclomatic complexity means the conservation result is not an artefact of how branches are counted. Disagreement would mean one arm's complexity is more deeply nested, which is a real finding about shape rather than amount.
Total NPath complexity
pmd_npath_total · = prediction: no arm difference · Nejmeh 1988, via PMD
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
The number of acyclic execution paths through a function. It is cyclomatic complexity's multiplicative cousin. Where McCabe adds across sequential branches, NPath multiplies, which is closer to the number of distinct behaviours a test suite would have to cover. Two sequential if-statements are 3 by McCabe and 4 by NPath. Ten of them are 11 versus 1024.
NP(seq) = NP(s1) × NP(s2) × …
pmd_npath_total = Σf ∈ F NP(body of f)
Where. The recursion is over the statement tree. Statements in sequence multiply, which is where the explosion comes from. A branch adds its arms. A statement with no control flow has NP 1. Loops and switch have their own rules in Nejmeh's original paper, which PMD implements.
In this harness. PMD, from the same single spawn as cognitive complexity.
How to read it. Because it multiplies, NPath detonates when decisions pile up in the same method. A conservation result that survives NPath is a strong one.
Total Halstead volume
halstead_volume · = prediction: no arm difference · Halstead 1977
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
Program size measured by vocabulary rather than by control flow. It asks how many distinct operators and operands there are and how often they appear. It has no concept of a branch at all, which is what makes it valuable here: if it agrees with the cyclomatic total, the conservation result is not a property of one family of measure.
where η = η1 + η2 and N = N1 + N2
Where. η1 is the number of distinct operators and η2 the number of distinct operands, so η is the vocabulary. N1 and N2 are the total occurrences of each, so N is the program length. Volume is therefore length times the bits needed to name one vocabulary item.
In this harness. The harness tokenises each file itself in placement.halstead. Symbolic operators and the control-flow keywords count as operators. Identifiers, numbers and type keywords count as operands. Comments are stripped. Every string and character literal is replaced by one placeholder operand before tokenising, so an arm cannot move its Halstead score by writing longer error messages. File volumes are summed.
How to read it. The fourth independent way of asking how much is there. Treat the units as arbitrary and compare the two lines.
Total Halstead effort
halstead_effort · = prediction: no arm difference · Halstead 1977
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
Halstead's own estimate of the mental work needed to write or understand the program. It is volume multiplied by difficulty, where difficulty grows as a small vocabulary of operands gets reused many times. It is more sensitive than volume, and it moves for reasons volume does not.
E = D · V
Where. D is difficulty. The first factor is half the operator vocabulary. The second is the average number of times each distinct operand is used, so a function that keeps reusing the same few variables scores as harder. V is the volume above. Effort is their product, summed over files.
In this harness. Computed in the same tokenising pass as volume.
How to read it. Read it as a more sensitive volume. A gap here with no gap in volume means one arm reuses its operands more heavily.
Total weighted methods per class (independent parser)
ck_wmc_total · = prediction: no arm difference · Chidamber & Kemerer 1994, via the CK tool (Aniche 2015)
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
The same total-complexity question, computed class by class, by a completely different tool with a completely different parser. This is a cross-check rather than a finding. Every other complexity number on this page comes from lizard. If a parser quirk were driving the conservation result, this is where it would show up.
ck_wmc_total = ΣC ∈ classes WMC(C)
Where. C ranges over every class the CK tool can parse. Chidamber and Kemerer left the per-method weight open. CK uses cyclomatic complexity, which is the conventional choice and matches the lizard side, so the two totals are directly comparable.
In this harness. CK is a Java program that parses source with Eclipse JDT, which is a full compiler front end. lizard uses its own lightweight parser. The harness runs CK once per checkpoint over the arm's source directories and reads its per-class CSV.
How to read it. Agreement with total cyclomatic complexity is the result you want, and it is a boring one. Disagreement means one of the two parsers is failing to read some file, which would make that file look perfect.
Maintainability Index, average file
mi_mean · ↑ higher is better · Coleman et al. 1994, in the SEI and Visual Studio rescaling
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
A composite index built from Halstead volume, cyclomatic complexity and lines of code, rescaled so that higher is more maintainable. It is the number a reviewer expects to see, and it combines all three families of measure, so it is a useful single check on whether aggregate maintainability differs at all before any placement argument is made.
mi_mean = mean over files of MI(file)
Where. V is that file's Halstead volume, CC its summed cyclomatic complexity and LOC its line count. The constants are Coleman's, fitted in 1994. The · 100/171 factor and the floor at zero are the later SEI rescaling into a 0 to 100 range, which is the form most tools report. It is computed per file and then averaged, not over the codebase as one blob.
In this harness. Computed in placement.maintainability_index from the harness's own Halstead pass and lizard's per-function complexity, grouped by file.
How to read it. It is location-blind by construction, which is precisely why it must not be the thesis statistic. It cannot see concentration.
Maintainability Index, worst file
mi_min · ↑ higher is better · Coleman et al. 1994
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
The Maintainability Index of the single worst file in the codebase. Unlike the average it cannot be diluted by adding good files, which makes it the version worth quoting.
Where. Same per-file MI as above. The minimum is taken over every production Java file with a positive volume and line count.
In this harness. Same pass as the mean.
How to read it. A falling line means the worst thing in the codebase is getting worse, whatever else is happening. That is usually the thing a maintainer will actually meet.
Function count
total_fns · · descriptive · lizard
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
How many functions exist in production Java. This is descriptive, and it is also the scale correction for several other metrics. A distributed architecture should climb here, because that is the mechanism it works by, not a finding about it.
Where. F is the same function population as the cyclomatic total. The population is production Java only, meaning the files matched by the arm's source_globs at that checkpoint's commit. Tests are excluded. OfficeFloor's YAML wiring is counted in a separate pool and is never folded into a Java denominator, because mixing them would let a wiring-based architecture dilute any per-line metric.
In this harness. Count of lizard's function records at the checkpoint commit.
How to read it. It matters because several concentration metrics are not scale-free. An arm with more units scores better on those for free. This line is how you check whether a concentration gap is a difference in shape or just a difference in count.
File count
total_files · · descriptive · lizard
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
How many production Java files exist. The same role as the function count, one level up, and the denominator for every file-based concentration index on this page.
Where. file(f) is the path the function was found in. Only files containing at least one parsed function are counted, which is worth knowing: a file the parser cannot read does not appear here either.
In this harness. Distinct file paths among lizard's function records.
How to read it. Read it beside any HHI or top-share figure. Those are not scale-free, so a rising file count lowers them for free.
Package count
total_packages · · descriptive · harness
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
How many Java packages the production code spans. It says whether new rules got their own home in the source tree or were filed into existing ones.
Where. package(f) is derived from the file's path below the source root, with the file name removed, which is the package a Java file in a conventional layout declares.
In this harness. Derived from file paths in placement._package_of rather than by reading package statements, so it reflects the directory structure a developer navigates.
How to read it. Descriptive. It is context for the package-level concentration index.
Production Java lines of code
java_loc · · descriptive · lizard nloc, summed
Line is the mean of ten runs. Band is one standard deviation. Click for full size.
Total production Java, excluding tests and excluding the YAML wiring. The raw size line. Every ratio metric on this page has this or a close relative as its denominator, so a surprising ratio is often a size story.
Where. nloc(f) is lizard's count of non-comment, non-blank lines in the function. Note that this sums function bodies, so file-level declarations, imports and field initialisers are not included. The population is production Java only, meaning the files matched by the arm's source_globs at that checkpoint's commit. Tests are excluded. OfficeFloor's YAML wiring is counted in a separate pool and is never folded into a Java denominator, because mixing them would let a wiring-based architecture dilute any per-line metric.
In this harness. Summed from lizard's per-function records. The YAML pool is counted into yaml_loc separately and the two are never added together.
How to read it. Both arms should grow. The interesting question is whether one grows faster for the same sixty rules, which would mean it needs more code to express the same behaviour.
No comments:
Post a Comment