scancode detects "Simplified BSD License" as "BSD-3-Clause AND BSD-2-Clause" in version 32.5.0 vs BSD-2-Clause in version 32.3.3

[SheriffManzoorF]

Description

scancode-toolkit version 32.5.0 detects the same "Simplified BSD License" header comment as BSD-3-Clause AND BSD-2-Clause (bsd-simplified AND bsd-new), whereas version 32.3.3 correctly detects it as just BSD-2-Clause (bsd-simplified).
The statement causing this detection is:

// Distributed under the Simplified BSD License.
// See README.md for details.

Source processed: https://github.com/russross/blackfriday/tree/master

Affected files

10 files under russross/:

README.md
block.go
block_test.go
html.go
inline.go
inline_test.go
latex.go
markdown.go
markdown_test.go
ref_test.go

How To Reproduce

Clone https://github.com/russross/blackfriday (master branch)
Run scancode against the checkout with both v32.3.3 and v32.5.0, e.g.:
scancode --license --json-pp result.json .
Compare the license_expression for the files listed above:
v32.3.3 → bsd-simplified (BSD-2-Clause)
v32.5.0 → bsd-simplified AND bsd-new (BSD-3-Clause AND BSD-2-Clause)

System configuration

For bug reports, it really helps us to know:

• OS: N/A (issue reproducible regardless of OS)
• scancode-toolkit versions compared: 32.3.3 vs 32.5.0
• Installation method: N/A

32.3.3.json
32.5.0.json

[SheriffManzoorF]

Hi @AyanSinhaMahapatra , I am tagging you I realized my earlier issue #4718 has somewhat similar behavior for which you might have been involved too?

[tirthraj01]

Hi @SheriffManzoorF , I have investigated this issue and submitted PR #5311 to add the missing license rules for the Simplified BSD notice referencing LICENSE.txt. It resolves the overlapping token match and eliminates the false positive bsd-new detection. Looking forward to your review!