Cannot collect package scan from an npm package.json without a version aka. "private"

[chinyeungli]

scancode package scan fail the extract information from the following package.json file

{ "name" :            "bson"
, "description" :     "A bson parser for node.js and the browser"
, "main":             "../"
, "directories" :   { "lib" : "../lib/bson" }
, "engines" :       { "node" : ">=0.6.0" }
, "licenses" :    [ { "type" :  "Apache License, Version 2.0"
                    , "url" :   "http://www.apache.org/licenses/LICENSE-2.0" } ]
}

[pombredanne]

Good catch... If I run this:
SCANCODE_DEBUG_PACKAGE_API=yes ./scancode -p --json-pp - package.json
this gives me extra debug info:

$ SCANCODE_DEBUG_PACKAGE_API=yes ./scancode -p --json-pp - package.json 
Setup plugins...
Collect file inventory...
Scan files for: packages with 1 process(es)...
[####################] 1                      
{
  "headers": [
    {
      "tool_name": "scancode-toolkit",
      "tool_version": "21.1.21",
      "options": {
        "input": [
          "package.json"
        ],
        "--json-pp": "-",
        "--package": true
      },
      "notice": "Generated with ScanCode and provided on an \"AS IS\" BASIS, WITHOUT WARRANTIES\nOR CONDITIONS OF ANY KIND, either express or implied. No content created from\nScanCode should be considered or used as legal advice. Consult an Attorney\nfor any legal advice.\nScanCode is a free software code scanning tool from nexB Inc. and others.\nVisit https://github.com/nexB/scancode-toolkit/ for support and download.",
      "start_timestamp": "2021-02-04T162740.058491",
      "end_timestamp": "2021-02-04T162741.025467",
      "duration": 0.9669885635375977,
      "message": null,
      "errors": [
        "Path: package.json"
      ],
      "extra_data": {
        "files_count": 1
      }
    }
  ],
  "files": [
    {
      "path": "package.json",
      "type": "file",
      "packages": [],
      "scan_errors": [
        "ERROR: for scanner: packages:\nERROR: Unknown error:\nTraceback (most recent call last):\n  File \"/home/pombreda/w421/scancode-toolkit-master/src/scancode/interrupt.py\", line 88, in interruptible\n    return NO_ERROR, func(*(args or ()), **(kwargs or {}))\n  File \"/home/pombreda/w421/scancode-toolkit-master/src/scancode/api.py\", line 298, in get_package_info\n    return dict(packages=[package.to_dict() for package in recognized_packages])\n  File \"/home/pombreda/w421/scancode-toolkit-master/src/scancode/api.py\", line 298, in <listcomp>\n    return dict(packages=[package.to_dict() for package in recognized_packages])\nAttributeError: 'NoneType' object has no attribute 'to_dict'\n"
      ]
    }
  ]
}Scanning done.
Some files failed to scan properly:
Path: package.json
Summary:        packages with 1 process(es)
Errors count:   1
Scan Speed:     8.63 files/sec. 
Initial counts: 1 resource(s): 1 file(s) and 0 directorie(s) 
Final counts:   1 resource(s): 1 file(s) and 0 directorie(s) 
Timings:
  scan_start: 2021-02-04T162740.058491
  scan_end:   2021-02-04T162741.025467
  setup_scan:licenses: 0.85s
  setup: 0.85s
  scan: 0.12s
  total: 0.97s

which is only mildly helpful

[nishakm]

I may have the same issue with the CLI:
tern-tools/tern#818 (comment)

[rnjudge]

@chinyeungli @pombredanne any update on this or idea when someone can take a look? We are not able to get package manifest data from scancode at the moment which impacts Tern's file license reporting. We run scancode like so: scancode -ilpcu --quiet --timeout 300 --json - /path/to/rootfs

[Divyansh2512]

@MaJuRG @pombredanne @chinyeungli I did a little research on this and it seems like a package.json file must have both 'name' and 'version' fields. Since there is no version field in this particular package.json file, that is why it is returning empty list for the packages field.

[Divyansh2512]

https://github.com/nexB/scancode-toolkit/blob/a381ec46451fef82911cb952685c8d41562d5873/src/packagedcode/npm.py#L268-L271
This is the original code.

[steven-esser]

@Divyansh2512 I think we should relax this constraint for npm packages and only check for existance of name.

@pombredanne thoughts?

[pombredanne]

@Divyansh2512 I think we should relax this constraint for npm packages and only check for existance of name.

That's a good approach. I like this. Eventually we may even relax it further in the future to accept no name and no version. for #2098 (comment) ... but for now, going with name only is a good way.

[pombredanne]

BTW, this is what I think are called "private" npms ... per https://docs.npmjs.com/cli/v6/configuring-npm/package-json#version

If you don't plan to publish your package, the name and version fields are optional.

[Divyansh2512]

SGTM! But judging from:-

I may have the same issue with the CLI:
tern-tools/tern#818 (comment)

@chinyeungli @pombredanne any update on this or idea when someone can take a look? We are not able to get package manifest data from scancode at the moment which impacts Tern's file license reporting. We run scancode like so: scancode -ilpcu --quiet --timeout 300 --json - /path/to/rootfs

It seems like there issue is not for package.json file. @nishakm @rnjudge can you please send us a sample file against which it is not listing the packages.

[steven-esser]

@Divyansh2512 Sure, but for now this issue is only in relation to npm packages (see first post by @chinyeungli). If the tern folks have other package manifest issues, we can handle that in a separate ticket.

[nishakm]

I haven't tested this yet. I will provide feedback shortly.