Recommended install method fails: No module named 'distutils.cmd'

[ohemelaar]

Description

Following the README's instructions to install using the recommended method fails with the following output.

* Configuring ScanCode for first use...
Traceback (most recent call last):
  File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/<redacted>/.local/share/virtualenv/wheel/house/pip-21.2.3-py3-none-any.whl/pip/__main__.py", line 29, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "/home/<redacted>/.local/share/virtualenv/wheel/house/pip-21.2.3-py3-none-any.whl/pip/_internal/cli/main.py", line 9, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "/home/<redacted>/.local/share/virtualenv/wheel/house/pip-21.2.3-py3-none-any.whl/pip/_internal/cli/autocompletion.py", line 10, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "/home/<redacted>/.local/share/virtualenv/wheel/house/pip-21.2.3-py3-none-any.whl/pip/_internal/cli/main_parser.py", line 8, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "/home/<redacted>/.local/share/virtualenv/wheel/house/pip-21.2.3-py3-none-any.whl/pip/_internal/cli/cmdoptions.py", line 23, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "/home/<redacted>/.local/share/virtualenv/wheel/house/pip-21.2.3-py3-none-any.whl/pip/_internal/cli/parser.py", line 12, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "/home/<redacted>/.local/share/virtualenv/wheel/house/pip-21.2.3-py3-none-any.whl/pip/_internal/configuration.py", line 27, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "/home/<redacted>/.local/share/virtualenv/wheel/house/pip-21.2.3-py3-none-any.whl/pip/_internal/utils/misc.py", line 42, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "/home/<redacted>/.local/share/virtualenv/wheel/house/pip-21.2.3-py3-none-any.whl/pip/_internal/locations/__init__.py", line 14, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "/home/<redacted>/.local/share/virtualenv/wheel/house/pip-21.2.3-py3-none-any.whl/pip/_internal/locations/_distutils.py", line 9, in <module>
ModuleNotFoundError: No module named 'distutils.cmd'
Traceback (most recent call last):
  File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/<redacted>/Downloads/scancode-toolkit-30.1.0/etc/thirdparty/virtualenv.pyz/__main__.py", line 168, in <module>
  File "/home/<redacted>/Downloads/scancode-toolkit-30.1.0/etc/thirdparty/virtualenv.pyz/__main__.py", line 164, in run
  File "/home/<redacted>/Downloads/scancode-toolkit-30.1.0/etc/thirdparty/virtualenv.pyz/virtualenv/__main__.py", line 18, in run
  File "/home/<redacted>/Downloads/scancode-toolkit-30.1.0/etc/thirdparty/virtualenv.pyz/virtualenv/run/__init__.py", line 32, in cli_run
  File "/home/<redacted>/Downloads/scancode-toolkit-30.1.0/etc/thirdparty/virtualenv.pyz/virtualenv/run/session.py", line 47, in run
  File "/home/<redacted>/Downloads/scancode-toolkit-30.1.0/etc/thirdparty/virtualenv.pyz/virtualenv/run/session.py", line 60, in _seed
  File "/home/<redacted>/Downloads/scancode-toolkit-30.1.0/etc/thirdparty/virtualenv.pyz/virtualenv/seed/embed/pip_invoke.py", line 23, in run
  File "/home/<redacted>/Downloads/scancode-toolkit-30.1.0/etc/thirdparty/virtualenv.pyz/virtualenv/seed/embed/pip_invoke.py", line 31, in _execute
RuntimeError: failed seed with code 1

Further runs show this error:

./scancode: line 122: /home/<redacted>/Downloads/scancode-toolkit-30.1.0/bin/scancode: No such file or directory

How To Reproduce

Tell us how to reproduce the issue.

 wget https://github.com/nexB/scancode-toolkit/releases/download/v30.1.0/scancode-toolkit-30.1.0_py39-linux.tar.xz
 tar xf scancode-toolkit-30.1.0_py39-linux.tar.xz 
 cd scancode-toolkit-30.1.0/
 ./scancode --help

or with beta release

 wget https://github.com/nexB/scancode-toolkit/releases/download/v31.0.0b3/scancode-toolkit-31.0.0b3_py39-linux.tar.xz
 tar xf scancode-toolkit-31.0.0b3_py39-linux.tar.xz 
 cd scancode-toolkit-31.0.0b3/
 ./scancode --help

System configuration

• Ubuntu 21.10
• Python 3.9.7
• scancode v30.1.0 - 2021-09-25 or v31.0.0b3 - 2022-04-30
• Downloaded from releases

[ohemelaar]

I could work around this issue by running sudo apt install python3-distutils before running ./scancode --help for the first time. So maybe this dependency should be documented in the README?

[pombredanne]

@ohemelaar Thank you for the report!
the missing sudo apt install python3-distutils is kinda weird since this is a standard library module, but Debian has always done weird things such as this.... let me think if there could be a better way..

Do you know if this is specific to Ubuntu 21?
What is the default Python version on Ubuntu 21?

Note that there is a fairly extensive issues in pypa/get-pip#124

[pombredanne]

I could work around this issue by running sudo apt install python3-distutils before running ./scancode --help for the first time. So maybe this dependency should be documented in the README?

Yes, it should be for sure... do you mind to propose a PR to update the installation doc? 😇

[pombredanne]

@ohemelaar gentle ping:

Do you know if this is specific to Ubuntu 21?
What is the default Python version on Ubuntu 21?

[pombredanne]

@ohemelaar based on https://bugs.launchpad.net/ubuntu/+source/python3.10/+bug/1940705 ... can you try without python3-distutils and the latest updates?

[ohemelaar]

Sorry for the delay!

What is the default Python version on Ubuntu 21?

The python version is 3.9.7.

@ohemelaar based on https://bugs.launchpad.net/ubuntu/+source/python3.10/+bug/1940705 ... can you try without python3-distutils and the latest updates?

I just tried on my setup and on a freshly installed 21.10 VM and encountered the same error. Indeed, the bug report seems to only mention python 3.10. I wanted to test on a 22.04 VM too but found out that scancode does not support python3.10 yet.

[camillem]

Hi,
There is a similar problem on Ubuntu 22.04 LTS, where Python 3 is 3.10.
It will be fixed by #2956 but in the meantime, it can be fixed by installing Python 3.8 from the "deadsnakes" ppa.

sudo apt update && sudo apt upgrade
sudo add-apt-repository ppa:deadsnakes/ppa -y
sudo apt install python3.8 python3.8-distutils

@pombredanne : I'd be happy to document that in a PR if you find it appropriate

[AyanSinhaMahapatra]

@camillem please do thanks! Sorry for the late reply.