Upgrade to support also Python 3.x

[pombredanne]

The time is now!
Some important reads:

• the official guide: https://docs.python.org/3/howto/pyporting.html
• a great extensive guide from @fedora-python https://portingguide.readthedocs.io/en/latest/process.html

[pombredanne]

And this likely means only Python 3.5 and up

[sschuberth]

Here's one more vote to support Python 3 as that would make debugging multiprocessing exceptions like issue #413 a bit easier.

[pombredanne]

At a high level the key issues I can think of are:

• unicode vs. bytes, since a good many functions handle low level bytes and not unicode and Py3 is unicode by default.
• native libraries using C/C++
• other third-party libraries
• doctests (with u'' in their expectations) or other similar tests

And in all cases, I want to support 2 and 3 at the same time with a single codebase which is not a big deal.

FWIW, most libraries should be OK except some natives. Upcoming ones (e.g. https://github.com/nexB/license-expression/ ) are also ok (I just ported that to Python 3)

[pombredanne]

For the 2 to 3 update, the first step will be effectively to gather wheels for every ScanCode-bundled dependencies

I do not know which one would be OK or not: this has to be tested

And eventually a Python 3 wheel --if not available-- can be built for all OSes/archs using https://github.com/pombreda/thirdparty-manylinux/ https://github.com/pombreda/thirdparty that are Travis and Appveyor build loops that create wheels and upload them to http://dl.bintray.com/pombreda/thirdparty/

Note that I have already started collecting Python 36 wheels there

And once all deps are available, the second step is to ensure that all the tests pass both on Py2 and Py3 and start making fixes, one at a time which will be the guide to make the port work

And BTW the minimum Python3 version we will support in ScanCode is 3.6.1.
Nothing before that 😉

[vivonk]

According to me, most helpful is this so why we are collecting python wheels on version dependent. Are you trying say that packages of python3 should be installed as well like it do in python 2 case(bin/python2.7) for virtualenv purposes?

[pombredanne]

@vivonk yes, that should be the way

[sschuberth]

More reasons to upgrade to Python 3.x, and version 3.7 in particular: https://docs.python.org/3/whatsnew/3.7.html#optimizations

[davidak]

You have 11 months left to migrate to python 3. Then python 2 will no longer be maintained.

https://pythonclock.org/

What is the status?

[pombredanne]

@davidak this is definitely in the plan for this year!
This is likely quite a bit of work especially if we want to keep support for Python 2 which may not be a good idea?

Help is definitely needed and welcomed. Would you be willing to chip in to help a bit if you have a few spare cycles?

Also, we are planning to participate in the Google Summer of Code this year and if we are accepted that would be a great student project. Mentoring help could be great there.