Not usable with commoncode 21.1.14

[danielthieleog]

Description

When using new commoncode release 21.1.14 calling scancode fails.
This was not the case with commoncode release 20.10.20.

Error message from scancode

Traceback (most recent call last):
File "/app/venv-scancode/bin/scancode", line 5, in
from scancode.cli import scancode
File "/app/venv-scancode/lib/python3.6/site-packages/scancode/cli.py", line 73, in
from commoncode.fileutils import PATH_TYPE
ImportError: cannot import name 'PATH_TYPE'

How To Reproduce

# Workdir is /app
pip install virtualenv \
    && virtualenv -p /usr/local/bin/python3.6 venv-scancode \
    && . venv-scancode/bin/activate \
    && pip install scancode-toolkit pyicu

virtualenv -p /usr/local/bin/python3.6 venv-scancode
. venv-scancode/bin/activate
    
/app/venv-scancode/bin/scancode

Error also occurs with python 3.9.1.

System configuration

Running on Linux in python:3.6 docker container.
Release 3.2.3 of scancode-toolkit.
Installed with pip.

[priv-kweihmann]

aboutcode-org/commoncode@0c646b7 removed that code, although that doesn't look perfectly right to me - The block starting from aboutcode-org/commoncode@0c646b7#diff-95c7b154fe632f7ace737d26910c5bc5b7d0a5947d09ffcd905fae446e58d588L96 shouldn't have been removed IMO

But to me that is rather a bug in commoncode

[JeroenKnoops]

Also happens in commoncode-21.1.21
PATH_TYPE can not be imported anymore.

[priv-kweihmann]

I hope aboutcode-org/commoncode#10 will fix the issue

[danielthieleog]

Not fixed yet. aboutcode-org/commoncode#10 is still open.

[Angi2412]

Not fixed yet. nexB/commoncode#10 is still open.

Did you find a workaround while the issue is not fixed?
Or should I just use an earlier release of scancode?

[JeroenKnoops]

I'm not sure how to use an earlier release...

[priv-kweihmann]

you can enforce earlier versions of commoncode (https://pypi.org/project/commoncode/20.10.20/ should work fine)

[Angi2412]

you can enforce earlier versions of commoncode (https://pypi.org/project/commoncode/20.10.20/ should work fine)

Thank you! That works for now 👍

[JeroenKnoops]

Sorry to bother again, but I don't seem to get that working.. How did you do this @Angi2412 ?

New vitualenv:

pip install commoncode==20.10.20
pip install scancode-toolkit[full]

Still uses 21.1.21

[priv-kweihmann]

pip install scancode-toolkit[full]
pip install --force commoncode==20.10.20

should do the trick

[JeroenKnoops]

Unfortunately this will result in:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
typecode 21.1.21 requires commoncode>=21.1.21, but you have commoncode 20.10.20 which is incompatible.
scancode-toolkit 3.2.3 requires chardet<4.0.0,>=3.0.0, but you have chardet 4.0.0 which is incompatible.
plugincode 21.1.21 requires commoncode>=21.1.21, but you have commoncode 20.10.20 which is incompatible.
extractcode 21.1.21 requires commoncode>=21.1.21, but you have commoncode 20.10.20 which is incompatible.
debut 0.9.8 requires chardet<4.0.0,>=3.0.0, but you have chardet 4.0.0 which is incompatible.

I will wait till the release... No worries.

[danielthieleog]

As workaround try to install old commoncode 20.10.20 together with the other packages, e.g.

pip install virtualenv \
    && virtualenv -p /usr/local/bin/python3.6 venv-scancode \
    && . venv-scancode/bin/activate \
    && pip install commoncode==20.10.20 scancode-toolkit extractcode pyicu

[Angi2412]

Sorry to bother again, but I don't seem to get that working.. How did you do this @Angi2412 ?

New vitualenv:

pip install commoncode==20.10.20
pip install scancode-toolkit[full]

Still uses 21.1.21

I use it in a requirements.txt file and install it in a Dockerfile.
Content of requirements.txt:

[...]
commoncode==20.10.20
scancode-toolkit[full]
[...]

Content of Dockerfile:

[...]
RUN python -m pip install --upgrade pip
RUN pip install -r requirements.txt
[...]

[danielthieleog]

Installing the current version with pip using scancode-toolkit[full] no longer showed the errors. It installed scancode_toolkit==21.2.9 with commoncode==21.1.21