Scancode is not filling the "packages" - attribute in the scanresult when a folder is existent, that in order, is before node_modules

[MarcelWorschech]

Description

There seems to be some problem with analyzing the installed packages when not following a certain structure:

The example given is based on an npm project:

Lets say we have a project called "test"
This project has one module installed (e.g. "ajv") via npm

1 folder and 2 files are present:

node_modules
package.json
package-lock.json

Scanning this project returns a scanfile where the package-attribute contains 6 packages (ajv and its dependencies)

If i however create a folder (doesnt matter if the folder is empty or not) and i name this folder "nn", the returned packages will shrink down to 1.
Naming the folder "np" instead, will return the packages as expected.

This somehow has to do with the order of the folder. If the Folder is alphabetically before the node_modules folder, it will screw up the result.
If the folder stands after the node_modules folder, it will work.

This bug is reproducable by following the described steps.
I don't know if this happens with other package managers also, i just tried this with npm.

System configuration

Scancode-version:
scancode-toolkit-v31.2.1

Scancode-command:
"../scancode-toolkit-v31.2.1/scancode -clpui -n 6 --json-pp "./scannerResult/scanner-result.json" "../test"

Model Name: MacBook Pro
Model Identifier: MacBookPro18,3
Model Number: Z15J00090D/A
Chip: Apple M1 Pro
Total Number of Cores: 10 (8 performance and 2 efficiency)
Memory: 32 GB
System Firmware Version: 8419.41.10
OS Loader Version: 8419.41.10

[pombredanne]

@MarcelWorschech Thank you for this report!

This is likely a bug alright. It may be the same as #3139 as reported @bwjohnson-ss

I created a test project at https://github.com/pombredanne/test-npm

I ran npm install and committed the whole thing.

Then using the latest SCTK develop at d64acdd I ran this:

• scancode-reindex-licenses
• scancode --package -n 6 --json-pp scanner-result.json.txt ../test-npm/ --yaml scanner-result.yml.txt

The results are attached.

scanner-result.json.txt
scanner-result.yml.txt

It seems to create correctly the 7 npms:

• 1 top level (my test-npm)
• 6 deps under node_modules

I wonder if the changes pushed by @JonoYang and @AyanSinhaMahapatra may have fixed it ? Or is the issue still there? Can you tell if the attached match your expectations?

[MarcelWorschech]

Hey,

sry for the late answer. The attached do match my expectations, but for the bug to occur you need to also have an additional folder present, ranking higher in order than node_modules ("nn" for example)