Scancode is not correctly scanning some licenses

[tarungunampalli1]

Description

We have a project dependency on mapbox-gl which contains this line in their package.json:
"license": "SEE LICENSE IN LICENSE.txt",
I'll link the LICENSE.txt file here
When I run scancode, I get licenseref-scancode-unknown-license-reference.

I also get unknown license reference for js-cookie and css-declaration-sorter, which have very normal license definitions, "license": "MIT", and "license": "ISC", respectively.

Finally, I get licenseref-scancode-unknown for an internal package with "license": "UNLICENSED", in its package.json.

How To Reproduce

Install any of the mentioned packages (mapbox-gl, js-cookie, or css-declaration-sorter) and run scancode

scancode --verbose --license --package --processes=4 --json-pp scan_results.json --include package.json --include *.txt .

System configuration

For bug reports, it really helps us to know:

• What OS are you running on? (Windows/MacOS/Linux)
• Linux
• What version of scancode-toolkit was used to generate the scan file?
• What installation method was used to install/run scancode? (pip/source download/other)
• pip

[tyllark]

We are encountering a similar problem with ts-jest incorrectly including a LicenseRef-scancode-free-unknown reference. This line (specifically open source license) seems to be the culprit.

[pombredanne]

@tyllark your problem is entirely different from @tarungunampalli1 . Do you mind to file a separate issue?
The issue in your case is about this rule https://github.com/aboutcode-org/scancode-toolkit/blob/develop/src/licensedcode/data/rules/free-unknown_88.RULE which is a "clue". @AyanSinhaMahapatra any idea why this show up in a detection then?

[pombredanne]

@tarungunampalli1 There are several issues:

1. the workspace from the package.json seems to be processed incorrectly
2. the license detection of the LICENSE.TXT is missing the reference to the the Mapbox TOS. See: Add Mapbox TOS license #4218 ... this software is entirely proprietary.
3. we do not deference the package.json declared license to LICENSE.TXT which is a bug in https://github.com/aboutcode-org/scancode-toolkit/blob/develop/src/packagedcode/licensing.py

@AyanSinhaMahapatra we have a regression there ^

@tarungunampalli1 at least the detection of "licenseref-scancode-unknown-license-reference" helps even if incorrect as there are multiple issues with the detection.

re:

Finally, I get licenseref-scancode-unknown for an internal package with "license": "UNLICENSED", in its package.json.

can you provide an example of how this package.json relates to and is located relative to the code of mapbox?

[pombredanne]

See also related:

• Improve npm package declared license detection #2488

[tarungunampalli1]

re:

Finally, I get licenseref-scancode-unknown for an internal package with "license": "UNLICENSED", in its package.json.
can you provide an example of how this package.json relates to and is located relative to the code of mapbox?

This isn't related to the mapbox code. My original post was listing out three different issues, one with mapbox, one with js-cookie and css-declaration-sorter, and one with the internal package. The internal package is just in our src/ directory.

[pombredanne]

@tarungunampalli1 re:

js-cookie and css-declaration-sorter

Do you mind to post links to the exact versions and possibly a scan using the latest ScanCode version using these options:
scancode -license --license-text --license-text-diagnostics --license-diagnostics --package --yaml scan.yml <your file> (YAML is easier to read, but pretty printed JSON is fine too)

[pombredanne]

@tarungunampalli1 re:

This isn't related to the mapbox code. My original post was listing out three different issues, one with mapbox, one with js-cookie and css-declaration-sorter, and one with the internal package. The internal package is just in our src/ directory.

Each issues is kind of unique when it comes to license detection and the devil is in the details .... so I jumped on mapbox because you had a link.