Undetected packages in docker scan

[chinyeungli]

I ran a Docker scan on its own (i.e., pipeline: analyze_docker_image, input: docker://ghcr.io/aboutcode-org/scancode.io:v34.9.5).

In layer 12, there are two packages that are not detected, and they are:

/opt/scancodeio/.venv/lib/python3.12/site-packages/beartype /opt/scancodeio/.venv/lib/python3.12/site-packages/beartype-0.19.0.dist-info

/opt/scancodeio/.venv/lib/python3.12/site-packages/normality /opt/scancodeio/.venv/lib/python3.12/site-packages/normality-2.6.1.dist-info

Other packages under /opt/scancodeio/.venv/lib/python3.12/site-packages are detected correctly.

There is no obvious reason why these packages are not detected as they also have the METADATA as others:

/opt/scancodeio/.venv/lib/python3.12/site-packages/beartype-0.19.0.dist-info/METADATA
/opt/scancodeio/.venv/lib/python3.12/site-packages/normality-2.6.1.dist-info/METADATA

[tdruez]

There is no obvious reason why these packages are not detected as they also have the METADATA as others

TL;DR: This is a bug on the ScanCode toolkit side, in PythonInstalledWheelMetadataFile.parse
Entered as aboutcode-org/scancode-toolkit#4174

---

>>> from scanpipe.pipes.scancode import scan_for_package_data

>>> scan_for_package_data("/scancode.io/.venv/lib/python3.12/site-packages/beartype-0.19.0.dist-info/METADATA")
({'package_data': []}, []) 

# Looking at another METADATA entry:
>>> scan_for_package_data("/scancode.io/.venv/lib/python3.12/site-packages/aboutcode_hashid-0.2.0.dist-info/METADATA")
({'package_data': [{'type': 'pypi', 'namespace': None, 'name': 'aboutcode.hashid', 'version': '0.2.0' ....

The scan_for_package_data returns nothing on beartype-0.19.0.dist-info/

Going deeper :

>>> from scancode.api import get_package_data

>>> get_package_data("/scancode.io/.venv/lib/python3.12/site-packages/beartype-0.19.0.dist-info/METADATA")
{'package_data': []}

Same issue when calling the toolkit api function directly.

>>> from packagedcode.pypi import PythonInstalledWheelMetadataFile

>>> location = "/scancode.io/.venv/lib/python3.12/site-packages/normality-2.6.1.dist-info/METADATA"
>>> results = PythonInstalledWheelMetadataFile.parse(location=location, package_only=False)
>>> print(list(results))

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/Volumes/nexB/repos/scancode.io/.venv/lib/python3.12/site-packages/packagedcode/pypi.py", line 380, in parse
    yield parse_metadata(
          ^^^^^^^^^^^^^^^
  File "/Volumes/nexB/repos/scancode.io/.venv/lib/python3.12/site-packages/packagedcode/pypi.py", line 1013, in parse_metadata
    extracted_license_statement, license_file = get_declared_license(metainfo=meta)
                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Volumes/nexB/repos/scancode.io/.venv/lib/python3.12/site-packages/packagedcode/pypi.py", line 1686, in get_declared_license
    license_file = lic.pop('file')
                   ^^^^^^^
AttributeError: 'str' object has no attribute 'pop'

Bug in the get_delcared_license function.

[tdruez]

Fxied in aboutcode-org/scancode-toolkit#4174