`unknown-license-reference` to complex paths are not followed

[uidp]

Description

https://github.com/Automattic/node-canvas with tag 3.1.2 is being identified as LicenseRef-scancode-unknown-license-reference AND MIT

is this a scancode issue, or is this node-canvas s fault?
There are a very similar issues:

• Obvious gpl-2.0 text is detected as unknown-license-reference #4387
• Obvious mit text is detected as unknown-license-reference #4481

How To Reproduce

git clone git@github.com:Automattic/node-canvas.git
git checkout 3.1.2
scancode --license --json-pp /tmp/results.json node-cookie-signature

results in

{
  "path": "node-canvas/Readme.md",
  "type": "file",
  "detected_license_expression": "mit AND (mit AND unknown-license-reference)",
  "detected_license_expression_spdx": "MIT AND (MIT AND LicenseRef-scancode-unknown-license-reference)",
  "license_detections": [
    {
      "license_expression": "mit",
      "license_expression_spdx": "MIT",
      "matches": [
        {
          "license_expression": "mit",
          "spdx_license_expression": "MIT",
          "from_file": "node-canvas/Readme.md",
          "start_line": 632,
          "end_line": 632,
          "matcher": "2-aho",
          "score": 100,
          "matched_length": 3,
          "match_coverage": 100,
          "rule_relevance": 100,
          "rule_identifier": "mit_27.RULE",
          "rule_url": "https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/rules/mit_27.RULE"
        }
      ],
      "identifier": "mit-3ab5a1cd-6f73-3105-a815-d0d7c55b1922"
    },
    {
      "license_expression": "mit AND unknown-license-reference",
      "license_expression_spdx": "MIT AND LicenseRef-scancode-unknown-license-reference",
      "matches": [
        {
          "license_expression": "mit",
          "spdx_license_expression": "MIT",
          "from_file": "node-canvas/Readme.md",
          "start_line": 638,
          "end_line": 653,
          "matcher": "2-aho",
          "score": 100,
          "matched_length": 161,
          "match_coverage": 100,
          "rule_relevance": 100,
          "rule_identifier": "mit.LICENSE",
          "rule_url": "https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/licenses/mit.LICENSE"
        },
        {
          "license_expression": "unknown-license-reference",
          "spdx_license_expression": "LicenseRef-scancode-unknown-license-reference",
          "from_file": "node-canvas/Readme.md",
          "start_line": 657,
          "end_line": 657,
          "matcher": "2-aho",
          "score": 100,
          "matched_length": 2,
          "match_coverage": 100,
          "rule_relevance": 100,
          "rule_identifier": "unknown-license-reference_see-license_1.RULE",
          "rule_url": "https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/rules/unknown-license-reference_see-license_1.RULE"
        }
      ],
      "identifier": "mit_and_unknown_license_reference-ca4b59e1-36db-1321-b1ca-a4fad99cffb0"
    }
  ],
  "license_clues": [],
  "percentage_of_license_text": 4.26,
  "scan_errors": []
}

System configuration

For bug reports, it really helps us to know:

• What OS are you running on? (Windows/MacOS/Linux)
  Fedora 42

• What version of scancode-toolkit was used to generate the scan file?

% scancode --version
ScanCode version: 32.2.0
ScanCode Output Format version: 3.2.0
SPDX License list version: 3.24

• What installation method was used to install/run scancode? (pip/source download/other)
  dnf

[AyanSinhaMahapatra]

Thanks for the report @uidp !

The issue is the license reference to a complex path at: https://github.com/Automattic/node-canvas/blob/master/Readme.md?plain=1#L657

See [license](src/bmp/LICENSE.md)

The license detection result (note that I used these extra CL options for diagnostics: --license-text --license-diagnostics --license-text-diagnostics):

"files": [
    {
      "path": "Readme.md",
      "type": "file",
      "detected_license_expression": "mit AND (mit AND unknown-license-reference)",
      "detected_license_expression_spdx": "MIT AND (MIT AND LicenseRef-scancode-unknown-license-reference)",
      "license_detections": [
        {
          "license_expression": "mit",
          "license_expression_spdx": "MIT",
          "matches": [
            {
              "license_expression": "mit",
              "license_expression_spdx": "MIT",
              "from_file": "Readme.md",
              "start_line": 632,
              "end_line": 632,
              "matcher": "2-aho",
              "score": 100.0,
              "matched_length": 3,
              "match_coverage": 100.0,
              "rule_relevance": 100,
              "rule_identifier": "mit_27.RULE",
              "rule_url": "https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/rules/mit_27.RULE",
              "matched_text": "The MIT License)",
              "matched_text_diagnostics": "The MIT License)"
            }
          ],
          "identifier": "mit-3ab5a1cd-6f73-3105-a815-d0d7c55b1922"
        },
        {
          "license_expression": "mit AND unknown-license-reference",
          "license_expression_spdx": "MIT AND LicenseRef-scancode-unknown-license-reference",
          "matches": [
            {
              "license_expression": "mit",
              "license_expression_spdx": "MIT",
              "from_file": "Readme.md",
              "start_line": 638,
              "end_line": 653,
              "matcher": "2-aho",
              "score": 100.0,
              "matched_length": 161,
              "match_coverage": 100.0,
              "rule_relevance": 100,
              "rule_identifier": "mit.LICENSE",
              "rule_url": "https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/licenses/mit.LICENSE",
              "matched_text": "Permission is hereby granted, free of charge, to any person obtaining a copy of\nthis software and associated documentation files (the 'Software'), to deal in\nthe Software without restriction, including without limitation the rights to\nuse, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of\nthe Software, and to permit persons to whom the Software is furnished to do so,\nsubject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS\nFOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR\nCOPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER\nIN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\nCONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.",
              "matched_text_diagnostics": "Permission is hereby granted, free of charge, to any person obtaining a copy of\nthis software and associated documentation files (the 'Software'), to deal in\nthe Software without restriction, including without limitation the rights to\nuse, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of\nthe Software, and to permit persons to whom the Software is furnished to do so,\nsubject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS\nFOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR\nCOPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER\nIN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\nCONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE."
            },
            {
              "license_expression": "unknown-license-reference",
              "license_expression_spdx": "LicenseRef-scancode-unknown-license-reference",
              "from_file": "Readme.md",
              "start_line": 657,
              "end_line": 657,
              "matcher": "2-aho",
              "score": 100.0,
              "matched_length": 2,
              "match_coverage": 100.0,
              "rule_relevance": 100,
              "rule_identifier": "unknown-license-reference_see-license_1.RULE",
              "rule_url": "https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/rules/unknown-license-reference_see-license_1.RULE",
              "matched_text": "See [license](",
              "matched_text_diagnostics": "See [license]("
            }
          ],
          "identifier": "mit_and_unknown_license_reference-ca4b59e1-36db-1321-b1ca-a4fad99cffb0"
        }
      ],
      "license_clues": [],
      "percentage_of_license_text": 4.26,
      "scan_errors": []
    }
  ]
}

Scancode can handle license references to LICENSE/ other legalese files (for example: see license in LICENSE.md or likewise) which are located right beside the origin file, or references to package ecosystemm specific locations. But arbitrary paths are not supported yet.

This could be fixed by:

• adding a new mit license rule with the whole license text with the reference from line 638 -> 657
• or support these kinds of arbitrary referenced paths when they occur in structured formats like in markdown here, which are parseable.