Skip to content

python_version markers using >, != or ~= are silently dropped from dependency extra_data #5309

Description

@Daksha1611

Description

get_python_version_os() in src/packagedcode/pypi.py silently drops the python_version environment marker for some comparison operators, because the list of accepted operators contains < twice and is missing >:

https://github.com/aboutcode-org/scancode-toolkit/blob/develop/src/packagedcode/pypi.py#L2281

python_version_operators = ['<', '>=', '==', '<=', '<']

< is duplicated where > was presumably intended, so > never matches. != and ~= are absent as well, though both are valid marker operators under PEP 508.

The value returned by this function becomes extra_data on the emitted DependentPackage:

extra_data = {}
if req.marker:
    platform = get_python_version_os(req.marker)
    if platform:
        extra_data = platform

So a dependency declared as requests; python_version > "3.8" is reported with an empty extra_data, while requests; python_version >= "3.8" is reported correctly. The extracted marker data ends up inconsistent and incomplete depending only on which operator the package author happened to use.

How To Reproduce

Calling the current develop implementation over parsed requirements:

from packvers.requirements import Requirement
from packagedcode.pypi import get_python_version_os

for spec in [
    'requests; python_version >= "3.8"',
    'requests; python_version > "3.8"',
    'requests; python_version != "3.8"',
    'requests; python_version <= "3.8"',
    'requests; python_version ~= "3.8"',
]:
    print(spec, '->', get_python_version_os(Requirement(spec).marker))

Actual output:

requests; python_version >= "3.8"  -> {'python_version': '>= 3.8'}
requests; python_version >  "3.8"  -> {}          <-- dropped
requests; python_version != "3.8"  -> {}          <-- dropped
requests; python_version <= "3.8"  -> {'python_version': '<= 3.8'}
requests; python_version ~= "3.8"  -> {}          <-- dropped

Expected: each of these should report the operator and version under extra_data["python_version"].

The same is visible end to end by scanning a requirements.txt that contains requests; python_version > "3.8" with:

scancode --package --json-pp - requirements.txt

and inspecting extra_data on the resulting dependency.

Suggested fix

Replace the duplicated < with the operators that are actually missing, covering the full set allowed for a marker in PEP 508:

python_version_operators = [
    '<', '<=', '!=', '==', '>=', '>', '~=', '===',
]

System configuration

  • What OS are you running on? Linux (x86_64)
  • What version of scancode-toolkit was used? 33.0.0rc1, develop at 5ebebf2
  • What installation method was used to install/run scancode? source checkout
  • Python version: 3.x

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions