Make the comma rules explain what they describe (#372) - #375
Merged
Conversation
Its regex was a bare comma, which reaches every comma in every script,
so seven glued-honorific CJK names classified under a rule whose prose
is "lone post-comma piece routes to suffix/title, not first". Of the
eight names it explained, that description fit exactly one.
They diff as {given, suffix}, a subset of this rule's fields, and
classify() takes the first rule whose regex matches and whose fields
are a superset. fix(cjk-comma-compound) -- the rule named for that
shape -- also matched, but both carry a name_regex, so they share a
tier and file order decided. This rule was written first.
Anchor it to Latin, at the U+0250 threshold _is_latin_only already
uses, anchored at both ends so a mixed name cannot match on its Latin
half. Measured: reach 236 -> 215, this rule now claims one name
('Andrews, M.D.'), and all seven moved to fix(cjk-comma-compound),
which grew 10 -> 17.
The gate is unmoved -- 108 intentional / 0 unexplained at 1.4.0, 90/0
at 2.0.0, 1/0 at 2.1.0. No name lost an explanation; only which rule
gives it changed. That is the whole point: #372 is about rules
explaining diffs they do not describe, not about diffs going unexplained.
_CORPUS_CLAIMS re-recorded, which is the one guard that fired -- the
forcing function working as designed. The rule's comment kept the
history of how the CJK rows got here and now says how they left.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #375 +/- ##
=======================================
Coverage 98.51% 98.51%
=======================================
Files 44 44
Lines 2895 2895
=======================================
Hits 2852 2852
Misses 43 43 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Review of the previous commit found it relocated the over-claiming
rather than fixing it. Measured: all seven names diff as exactly
{given, suffix} with `family` byte-identical -- no order flip happens,
because the comma already fixes family/given order on both sides. But
fix(cjk-comma-compound) exists for the UNION, comma routing AND the
flip together, which is why its fields include `family`. Its own
comment says so. The seven landed there on a fields superset and file
order: the same field-shape coincidence #372 was filed over, moved one
rule along.
Add fix(cjk-comma-honorific-peel) for what actually happens -- #312's
honorific peeling off the post-comma given name into `suffix`, family
untouched -- with fields = ["given", "suffix"]. The field list is what
separates it from the compound rule, not its position: the union rows
move `family`, which is outside these fields, so they cannot match it
however early it sits. It is written first anyway so the narrower rule
is also the earlier one.
Now each of the three comma rules explains only what it describes:
fix(comma-family) 1 'Andrews, M.D.'
fix(cjk-comma-honorific-peel) 7 the peels
fix(cjk-comma-compound) 10 the unions, back to its original set
Gate unmoved at 108/0, 90/0, 1/0. Two rosters fired and were recorded:
_SPAN_BEARING_RULES (the rule carries the script class) and
_CORPUS_CLAIMS.
Three prose sites corrected, all found by review. Two were invalidated
by the previous commit: fix(cjk-comma-compound) still said the
{given,suffix} rows match fix(comma-family) "purely on file order", and
fix(suffix-routing) still pointed at a note about a split that no
longer exists. The third was inherited and had been wrong before this
branch: '田中さん, V.' is not a parity row. It diffs {family, suffix}
and reaches classify(); it reads as parity only under
Policy(lenient_comma_suffixes=False), which the corpora never use.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every other guard here measures a rule ALONE. _CORPUS_CLAIMS records what a regex reaches; the gate counts names. Neither can see which rule wins a contest -- which is the property #372 is about, and why seven names sat on fix(comma-family), whose prose describes none of them, for months with every guard green and the gate reporting 108/0 throughout. _CROSS_RULE_WINNERS records classify()'s answer for the ten names whose boundaries this file argues about, keyed by (name, diff shape). The diff shapes are measured against the 1.4.0 wheel, not guessed. 42 corpus names are reachable by two or more same-tier rules; these are the ones with an argument attached, and pinning the argument is cheaper than re-deriving it. Proven by mutation, both directions the reviewers named: drop Hangul from the peel rule's class -> 1 failed (three Korean names fall through to fix(suffix-routing), a fields-only catch-all about two-token Latin names; the gate stays 108/0 and reports nothing) move the peel rule after the compound rule -> 1 failed a pure reorder, no regex or fields change. Measured across the whole suite: this is the ONLY test that fails. _CORPUS_CLAIMS is order-blind by construction, so before this pin a reorder could silently reattribute names with 3217 tests green. Note 'Andrews, M.D.' diffs on the same {given, suffix} shape as the seven peels -- only the CJK lookahead separates them, so it is the row that fails if that lookahead is ever dropped. A failure here is not automatically a regression: a rule narrowed correctly may send its names somewhere better. It means someone has to look, which is the point. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The first of #372's confirmed over-claiming instances, resolved. Three commits: the narrowing, the rule the narrowing turned out to need, and the pin that would have caught the whole thing.
The bug
fix(comma-family)'sname_regexwas a bare,, reaching every comma in every script. It explained 8 corpus names; its prose — "lone post-comma piece routes to suffix/title, not first" — described one. The other seven were CJK glued-honorific names, winning on nothing but file order over rules that carry aname_regexand therefore share its precedence tier.What it took
1. Anchor
fix(comma-family)to Latin (34f573d), at the U+0250 threshold_is_latin_onlyalready uses, anchored at both ends so a mixed-script name cannot match on its Latin half.2. Give the seven a rule that describes them (
e308987). Review caught that step 1 relocated the problem: the seven diff as{given, suffix}withfamilybyte-identical — no order flip — whilefix(cjk-comma-compound)exists for the union of comma routing and the family-first flip, which is why its fields includefamily. Its own comment says so. They landed there on a fields superset and file order: the same coincidence #372 was filed over, one rule along.So
fix(cjk-comma-honorific-peel)now names what actually happens — #312's honorific peeling off the post-comma given name intosuffix— withfields = ["given", "suffix"]. The field list is what separates it from the compound rule, not its position: the union rows movefamily, outside these fields, so they cannot match it however early it sits.3. Pin who explains what (
45f74c8)._CROSS_RULE_WINNERSrecordsclassify()'s answer for the ten contested names, keyed by(name, diff shape), with the diff shapes measured against the 1.4.0 wheel rather than guessed.Result
fix(comma-family)Andrews, M.D.fix(cjk-comma-honorific-peel)fix(cjk-comma-compound)familymovesfix(cjk-comma-compound)returned to exactly the 10 names it had before any of this — not aimed for, it fell out of drawing the boundary by fields.Gates unmoved throughout:
108 intentional / 0 unexplainedat 1.4.0,90/0at 2.0.0,1/0at 2.1.0. Suite 3218.Why the pin earns its place
Every other guard measures a rule alone.
_CORPUS_CLAIMSrecords what a regex reaches; the gate counts names. Neither can see which rule wins a contest — which is exactly how seven names sat on the wrong rule for months with every guard green.Proven by mutation, both directions review identified:
fix(suffix-routing), a fields-only catch-all about two-token Latin names. The gate stays108/0and reports nothing. Now caught._CORPUS_CLAIMSis order-blind by construction.A failure here is not automatically a regression — a rule narrowed correctly may send its names somewhere better. It means someone has to look, which is the point.
Prose corrections
Three sites, all found by review. Two were invalidated by commit 1:
fix(cjk-comma-compound)still said the{given,suffix}rows matchfix(comma-family)"purely on file order", andfix(suffix-routing)still pointed at a note about a split that no longer exists. The third was inherited and wrong before this branch:'田中さん, V.'is not a parity row — it diffs{family, suffix}and reachesclassify(), reading as parity only underPolicy(lenient_comma_suffixes=False), which the corpora never use.Still open on #372
fix(suffix-routing). It is the onlyfields-only rule, so giving it any regex moves it between precedence tiers and reshuffles globally — a different problem, worth its own measurement. Ambiguity reporting and the specificity floor remain declined on the evidence recorded in #373.🤖 Generated with Claude Code