Raised as "an implementation returning run for every file satisfies every rule in the spec". That is true of the rule text — F-Total is conditional ("If every declarator succeeded") and nothing obliges an implementation to succeed at anything — and false of conformance: 114 of 127 fixtures assert at least one fold verdict with expected exports or values, and spec/fixtures.test.ts gates rule-to-fixture coverage in both directions.
So the floor exists and is deliberate. Two things follow.
It is undocumented. Nothing in spec/README.md says the fixtures are the floor. A reader auditing only the normative text reaches the original conclusion, correctly.
It floors the wrong axis. spec/fixtures/ is organised one directory per rule identifier, so the set is exactly "constructs the spec names", assembled by the people who named them. 65 of 127 are whole-build and exercise multi-file graphs, taint and per-file verdict maps, which is more than construct-level coverage — but every one is still a fixture written to exercise a rule. An implementation passing all 127 has shown it folds every construct the spec enumerates in combinations the authors thought to combine. It has shown nothing about the distribution of constructs in code nobody here wrote.
The corpus is the only instrument that measures that, and its external section is one checkout: jhgaylor/infisical-chant @ 91cdf130 — 6 entries, 31 files, 13 comparable, 13 agreed, 5 both fold, 18 no host. 13/13 agreement on 42% of one repository's files.
That number appears nowhere on the site. corpus.md says the external rows "sit in their own section of the report and never inside the totals", which is a policy and not a reason. The consequence is that the site's headline (440/440) rests entirely on a corpus written by one side of the comparison, which is the standing objection, and the one artifact that answers it is unpublished.
Suggested: publish it with its denominator visible. The honest headline is "one external codebase, 13 comparable files, no disagreements", and that is worth more than silence.
Raised as "an implementation returning
runfor every file satisfies every rule in the spec". That is true of the rule text —F-Totalis conditional ("If every declarator succeeded") and nothing obliges an implementation to succeed at anything — and false of conformance: 114 of 127 fixtures assert at least onefoldverdict with expected exports or values, andspec/fixtures.test.tsgates rule-to-fixture coverage in both directions.So the floor exists and is deliberate. Two things follow.
It is undocumented. Nothing in
spec/README.mdsays the fixtures are the floor. A reader auditing only the normative text reaches the original conclusion, correctly.It floors the wrong axis.
spec/fixtures/is organised one directory per rule identifier, so the set is exactly "constructs the spec names", assembled by the people who named them. 65 of 127 are whole-build and exercise multi-file graphs, taint and per-file verdict maps, which is more than construct-level coverage — but every one is still a fixture written to exercise a rule. An implementation passing all 127 has shown it folds every construct the spec enumerates in combinations the authors thought to combine. It has shown nothing about the distribution of constructs in code nobody here wrote.The corpus is the only instrument that measures that, and its external section is one checkout:
jhgaylor/infisical-chant@91cdf130— 6 entries, 31 files, 13 comparable, 13 agreed, 5 both fold, 18 no host. 13/13 agreement on 42% of one repository's files.That number appears nowhere on the site.
corpus.mdsays the external rows "sit in their own section of the report and never inside the totals", which is a policy and not a reason. The consequence is that the site's headline (440/440) rests entirely on a corpus written by one side of the comparison, which is the standing objection, and the one artifact that answers it is unpublished.Suggested: publish it with its denominator visible. The honest headline is "one external codebase, 13 comparable files, no disagreements", and that is worth more than silence.