Dev utility: Create release scripts to build archives

[pombredanne]

One archive for each OS/Python combo should be created and each should have its exact set of pinned requirements included in the built archive, no more and no less.

[pombredanne]

See #2117

[pombredanne]

So the way this would typically work out would be IMHO this:

1. there is a single MANIFEST.in file that excludes tests/ and docs and includes the thirparty/
2. somehow there is a script that is launched and that accepts as option something like --os window or linux or macox, --arch 32 or 64 and --python 36, 37, etc ... these should normally be the same value as the wheel tags. And also accept a requirements file as an input.
3. This script will first wipe clean the thirdparty/ directory
4. Then it will download there just the set of wheels, sdists and ABOUT files that are needed for the options that were chosen. These wheels should be selected exactly from the provided requirement files and fetched from our own "github-repo-release-Pypi-repo" and saved in the thirdparty/ dir
5. finally the script generates an a python setup.py sdist that will include the current thirdparty and possibly renames that file so it contains the proper os/arch/python "tags" it its archive name.

Another way can work too. That's one of the ways :)

[Abhishek-Dev09-zz]

@altendky : Your suggestion are also welcome here.

[Abhishek-Dev09-zz]

@pombredanne : Only sdist/wheels are available in thirdparty not ABOUT,NOTICE because we are using pip-download here, that will be integrated with romp because romp will provide matrix for python/os/arch.

Our command will something like this romp --platform linux --interpreter cpython --version 3.6 --version 3.7 --version 3.8 --platform windows --platform macos --architecture x86_64 --command 'python setup.py --quiet --use-default-version clean --all sdist --formats=bztar,zip bdist_wheel' --artifact-paths '*.whl' --artifact 'artifacts.tar.gz' --build-request-url 'https://dev.azure.com/Abhiromp/romp/_apis/build/builds?api-version=5.0' --definition-id 6 --username Abhiromp --archive-path '.' --verbose

[pombredanne]

The new https://github.com/nexB/scancode-toolkit/blob/1bddb92c70abcf26cf9435061e429bae62cbe040/etc/release/scancode-create-release.sh script creates these alright.