Repository navigation
MultilineStrings miscounts lines in UTF-8 text, and VariableAnalysis flags named by-reference arguments - #398
Conversation
Watson-Branch: #397
…unction reads as undefined.
There was a problem hiding this comment.
🔄 Changes Requested
Two small blockers, both in code this PR added. Everything else is in good shape, and all 13 acceptance criteria are met.
Issues Found
-
The new failure branch in
StructuredText::lineMatcheshas no test.CleanCode/Support/StructuredText.php:100returns 0 whenpreg_match_allfails. Nothing can force that failure.tests/PregOverrides.phpgained apreg_match_alloverride only in theSniffs\Stringsnamespace.StructuredTextlives inMikeBronner\CleanCode\Support, soPregFailurecannot reach it. If someone changes line 100 toreturn $count;, the suite stays green. A real PCRE failure would then throw aTypeErrorunderstrict_typesand crash phpcs.
Missing test: add aMikeBronner\CleanCode\Supportpreg_match_alloverride totests/PregOverrides.php, gated bypregArmed. Then write a unit test intests/Support/that armsPregFailureforpreg_match_alland callsisMarkdown,isConfigandisSerialisedon text that is structured. It asserts that each returns false and that no PHP diagnostic is raised.
This repo bounced the same gap before: unexercised defensive branches inMemberOrderingSniff(PR #291). -
The new fixtures carry comments that no linter requires. The rule is no comments in code unless the project's linter or config requires them. This holds even where existing files are full of comments.
phpcs.lint.xmlexcludes*/fixtures/*, and no sniff under test reads comment tokens, so none of these is needed. That covers about 20//lines across all nine new fixtures. Examples:MultilineStringsSniff/utf8-at-limit.php("Greek upsilon, U+03C5, encodes as CF 85."),VariableAnalysisSniff/named-by-reference.php, andMultilineStringsSniff/invalid-utf8.php. The last one also quotes"\xCF", which reads as if the fixture used escapes.
Fix: delete them, and update the line and column expectations that move. Where a case needs explaining (the invisible U+2005 spaces, the raw Latin-1 bytes), carry it in the test name or the dataset key.
What's Good
LineBreaksis correct. In valid UTF-8,0xC2is only ever a lead byte, so\xC2\x85matches a real U+0085 and never the0x85insideυ,Ņor U+2005. CRLF counts once because it is the first alternative. Keeping byte mode instead of/uis a good call, because/uwould have changed what\wand\smatch inStructuredText.- The sniff no longer takes a failed count as zero breaks, and that path is tested through
PregFailure. - Fixtures assert exact lines, columns and messages, and they include the real-newline controls.
- The
VariableAnalysissubclass keeps the upstream sniff code, so existing excludes and ignore comments keep working. It also fails safe on every unresolved name. - AC3 is met by a different route than its wording.
StringLiteraldecodes only\r\n,\n,\rand\t(CleanCode/Support/StringLiteral.php:85), so"\xCF"never reaches the invalid-UTF-8 path. Raw bytes are the only way to exercise it. Theiconv_strlennotice comes from the PHPCS tokenizer, it happens onmaintoo, and no CleanCode code callsiconv. So filtering it out of the assertion hides nothing from the code under test. - CI is green on PHP 8.3, 8.4 and 8.5.
📋 Non-blocking follow-ups
namesReferenceParameterdoes not checkReflectionFunction::isInternal()(CleanCode/Overrides/VariableAnalysis/Sniffs/CodeAnalysis/VariableAnalysisSniff.php:84-95).ArrayAccessorsSniff.php:423-426already does this. Adding it would stop a function loaded in the phpcs process from standing in for a project function. Disposition: Noted, not tracked.- The test "reports both cases through the upstream sniff alone" (
tests/Ruleset/UndefinedVariableTest.php:181) only checks that some violation appears. PinningUNDEFINED_VARIABLEat exact lines would make it stronger. The fixtures leave room for nothing else, so this is hardening, not a hole. Disposition: Noted, not tracked.
(Watson: fix the two blockers above. The follow-ups are optional. Do them if they are cheap while you are in there, or skip them.)
Please address the above and re-request review.
A preg_match_all override in the Support namespace lets PregFailure reach StructuredText. The new test checks that isMarkdown, isConfig and isSerialised return false, with no PHP diagnostic, when the match fails.
No linter requires them. The line expectations move with them, and the test names now say where a fixture holds invisible U+2005 spaces or raw Latin-1 bytes.
|
Both blockers are fixed, including every unit-belonging finding. I skipped the unrelated cosmetics.
The two non-blocking follow-ups ( |
There was a problem hiding this comment.
🔄 Changes Requested
Both blockers from the last round are fixed. One small blocker is left, in a doc paragraph this PR added. All 13 acceptance criteria are still met.
Issues Found
- The new paragraph in the multiline-strings reference breaks the file's wrap.
resources/boost/skills/clean-code/references/code-style-multiline-strings-heredoc.md:34is 110 characters ("…is never mistaken for NEL. Text that is not valid UTF-8 is read byte"). Lines 36 and 38 then wrap short, at 32 and 27 characters. The rest of this file, and nearly every file in that folder, wraps at about 80. The paragraph reads as if two lines were joined by an edit.
Fix: re-wrap lines 30-38 at about 80 characters. No wording change is needed.
What's Good
- Round-1 blocker 1 is fixed. The
MikeBronner\CleanCode\Supportpreg_match_alloverride intests/PregOverrides.phpis loaded bytests/bootstrap.phpbefore any test runs, so it reachesStructuredText::lineMatches.tests/Support/StructuredTextTest.phpforces the failure and checks thatisMarkdown,isConfigandisSerialisedreturn false with no diagnostic. The positive cases in the same file show the texts really are structured, so the test can fail. - Round-1 blocker 2 is fixed. The new fixtures hold no comments, and no new comment appears anywhere in the diff. Every line and column expectation was checked by hand against the trimmed fixtures, and they all line up. The test names now carry the invisible U+2005 spaces and the raw Latin-1 bytes.
- CI is green on PHP 8.3, 8.4 and 8.5.
- The earlier notes on
LineBreaks, the subclass approach and the AC3 raw-byte route still hold.
Checked and not blocking: the switch from preg_match to preg_match_all in StructuredText does not add new regex cost. main already had the same (?:^|\R)\s* shape, and on text that does not match, both functions try every offset. The AC4 sniff-level fixtures are the right tier, because they also prove that StringLiteral decodes the \n escapes. tests/Support/LineBreaksTest.php already covers the 0x85 rule at unit level.
📋 Non-blocking follow-ups
tests/Ruleset/UndefinedVariableTest.php:179-181checks only that the upstream run reports something. That run loads only the upstream sniff, and the fixtures are clean apart from the bug, so nothing else can pass it. PinningUNDEFINED_VARIABLEat exact lines would still make it more exact. Disposition: Noted, not tracked.namesReferenceParameterdoes not checkReflectionFunction::isInternal(), from round 1. Disposition: Noted, not tracked.
(Watson: fix the one blocker above. The follow-ups are optional. Do them if they are cheap while you are in there, or skip them.)
Please address the above and re-request review.
|
I fixed the blocker and every finding that belongs to this unit. I skipped the unrelated cosmetics.
The two non-blocking follow-ups ( |
There was a problem hiding this comment.
✅ Approved
The round-2 blocker is fixed, and a full re-review of the current head found nothing new. All 13 acceptance criteria are met.
Review Summary
- Round-2 blocker is fixed. The new paragraph in
code-style-multiline-strings-heredoc.mdnow wraps at 80 like the rest of the file. The CONTRIBUTING paragraph was re-wrapped the same way. Neither commit changes any wording. - Each acceptance criterion is met. Two were met by a different route than their wording, and both were noted in round 1. AC3 uses raw bytes because
StringLiteraldoes not decode\xCF. Both of its cases sit in one fixture file, and the test pins each of them. - One possible blocker was checked and dropped.
StructuredText::lineMatchesreturns 0 whenpreg_match_allfails (CleanCode/Support/StructuredText.php:100). That is not a line-break count. It is a "does any line match" check, and a failure means "not structured", which is howmainalready behaved. AC4 asks for exactly that. AC3's "never zero breaks" rule covers the sniff's own count, andMultilineStringsSniffreturns early onfalsethere.tests/Support/StructuredTextTest.phppins the failure path. - The VariableAnalysis override runs no analysed code.
function_existsdoes not autoload, andReflectionFunctionexecutes nothing. - No comments or docblocks were added. No
phpcs:ignorewas added. No mbstring function is used. - CI is green on PHP 8.3, 8.4 and 8.5.
- Everything belonging to the coherent unit is clean.
📋 Non-blocking follow-ups
tests/Ruleset/UndefinedVariableTest.php:179-182checks only that the upstream-only run reports something. PinningUNDEFINED_VARIABLEat exact lines would make it more exact. Disposition: Noted, not tracked.namesReferenceParameterdoes not checkReflectionFunction::isInternal(). This carries over from round 1. Disposition: Noted, not tracked.
Ready for @mikebronner to merge.
Summary
Implements #397. Two sniffs reported errors on correct code:
MultilineStringscounted bytes in UTF-8 text, and the bundledVariableAnalysissniff did not understand named by-reference arguments or nested arrow functions.Changes
Part 1: line breaks in UTF-8 text
CleanCode/Support/LineBreaks. It gives the line-break pattern for a text. For valid UTF-8 the pattern is LF, CRLF, CR, vertical tab, form feed and NEL, and NEL counts only as its own encodingC2 85. A0x85byte insideυ,Ņor U+2005 is no longer a break. For text that is not valid UTF-8 the pattern stays the old byte-mode\R.MultilineStringscounts with it. If the count itself fails, the sniff reports nothing. It never takes a failed match as zero breaks.StructuredTextbuildsMARKDOWN_LINE,INI_SECTIONandSETTING_LINEwith the same pattern, at all four call sites (isSerialisedincluded).line-break-kinds.phppins all three.uflag: PHP turns on Unicode character properties with/u, so\wand\swould start to match Greek letters and Unicode spaces. That changes whatStructuredTexttreats as a key or a space, which is outside this issue. The patterns stay in byte mode, and only the break token changes.Part 2:
VariableAnalysis.CodeAnalysis.VariableAnalysisCleanCode/Overrides/VariableAnalysis/Sniffs/CodeAnalysis/VariableAnalysisSniff.php. Its path repeats the upstream shape, so PHP_CodeSniffer gives it the upstream code. The reported code staysVariableAnalysis.CodeAnalysis.VariableAnalysis.UndefinedVariable, and existing excludes and ignore comments keep working.FunctionCalls::isGlobalFunctionCall), PHP knows the function, its reflected parameter with that name is by-reference, and the variable is the whole argument. Positional arguments still go to the upstream code unchanged. An unknown parameter name, a method call, or an unknown function is still a read.CleanCode/ruleset.xmlrefers to the subclass by path, in place of the upstream code. The<type>,<exclude>s and<properties>are unchanged.composer.jsonis unchanged: no newer upstream release is needed.Upstream status (AC10):
sirbrillig/phpcs-variable-analysisv3.0.0 is the newest tag, andtrunkis identical to it in both affected files. Upstream keys a named argument by its name, so its positional by-reference check never matches it, and it has no option for named arguments. For nested arrow functions, upstream #329 fixed a different shape (a read after a nested arrow function). Reading an outer arrow function's parameter inside an inner one is still reported ontrunk, and there is no option for it. So this PR takes the subclass approach. Reporting both upstream is a separate step.Docs: the multiline-strings reference states what a line break is. The UndefinedVariable reference and the README describe the subclass. CONTRIBUTING documents
CleanCode/Overrides/.Acceptance Criteria
utf8-at-limit.php: three strings at exactly three lines, with severalυ,Ņand U+2005 per line. No error.utf8-over-limit.php: the same strings at four lines. One error each, and the message states 4.line-break-kinds.php: LF, CRLF and CR each count once. U+0085 counts, and U+2028 and U+2029 do not.invalid-utf8.php: the over-limit string is reported as 5 lines, which is the byte-mode count including a lone0x85. The at-limit string is not reported. A forced count failure reports nothing and raises no diagnostic. Note:StringLiteraldecodes only\n,\r\n,\rand\t, so an escape such as"\xCF"reaches the sniff as four ASCII characters and cannot exercise this path. The fixtures carry raw Latin-1 bytes instead. The PHPCS tokenizer raises aniconv_strlen()notice while it measures columns in such a file, before any sniff runs and onmaintoo. Thephpcscommand line does not show it. The tests assert no diagnostic other than that one.RequireHeredocForStructuredText/utf8-line-breaks.php: one case per pattern, withυdirectly before markdown, an INI section and setting lines. None is reported. The three controls, with a real newline in that place, are reported.invalid-utf8.phpis classified byte by byte, as before.named-by-reference.php:preg_match,preg_match_all,str_replace,similar_textandparse_str, with the by-reference argument first, in the middle and last, one mixed call, and one fully qualified call. No error.nested-arrow-functions.php: two levels, three levels, an arrow function in a closure withuse, a method local, and a read inside a string. No error.failing.phpkeeps the same lines and columns.still-undefined.phpreports a by-value named argument, a read before the call, another undefined variable in the same call, an unknown parameter name, a by-value parameter name on another function, a method call, an unknown function, a non-variable argument, and a nested arrow function reading nothing defined. The unknown name raises no PHP error.VariableAnalysiscodes are untouched, and theUnusedLocalVariabletests pass unedited.02049c2(mainat4dfd55aplus the empty start commit). The new tests, run against that source (test names as they were at that time):utf8-at-limit.phpreported 3 errors andutf8-line-breaks.phpreported 6.named-by-reference.phpreported 16 errors, andnested-arrow-functions.phpreported 6. Every genuine report instill-undefined.phpappeared before as well. That test failed only on the extra false positives this PR removes. AC2's Greek strings were reported before as well, but with counts of 14, 16 and 11 lines. The LF, CR, CRLF, NEL and invalid-UTF-8 cases give the same result before and after.README.md, both named references and CONTRIBUTING are updated. No other statement in the repo outsidevendor/was false.composer lintandcomposer lint:selfreport no errors. No newphpcs:ignore. Nombstringfunction is used.Test Plan
tests/Support/StructuredTextTest.phpforcespreg_match_allto fail insideStructuredText.isMarkdown,isConfigandisSerialisedthen return false with no PHP diagnostic. Changing the fallback toreturn $count;turns all three cases red with aTypeError.LineBreakspatterns, the outer arrow function lookup, the global-call check, the whole-argument check, the by-reference check, thefunction_existscheck, the named-argument lookup, and the failed-count guard.vendor/bin/phpcs --standard=CleanCoderun by hand on each new fixture gives the same results.Fixes #397