Please clarify what the scancode-toolkit*-mini* distribution is

[sschuberth]

I'm have a hard time understanding this:

https://github.com/nexB/scancode-toolkit/blob/a29ae817d95296146ba8ac5bd94ed043d541cdc6/setup-mini.cfg#L7

pre-built binary dependencies

What exactly are these? To me, Python bytecode is also binary, so any wheel for a Python package is a pre-built binary dependency in my view. And if "pre-built binary dependencies" are omitted with mini, what are they being substituted with? Because their functionality must somehow be replaced.

Asking differently, what's the difference between using the mini distribution, and installing (full) scancode-toolkit with pip install --no-binary=:all:?

These are instead installed separately

How?

or with the extra_requires scancode-toolkit-mini[full]

This is even more confusing: What's the difference between installing scancode-toolkit-mini[full] and scancode-toolkit in the first place?

[pombredanne]

The mini distribution is a wheel with fewer dependencies that is only useful iff:

• You want to integrate ScanCode as a library that you call directly from Python,
• You only need a subset of the features: no file type detection, no archive detection or extraction, no extensive package detection,
• You will manage dependencies yourself,
• You accept that things will not work as tested because of the caveats above. And because this DOES NOT pass the scancode tests.
• This is really only a convenience build for integrators

You wrote:

What exactly are these? To me, Python bytecode is also binary, so any wheel for a Python package is a pre-built binary dependency in my view.

There is no Python bytecode in our wheels and in most wheels in general, only Python source code and in some case we also have native binaries pre-built from C/C++.

And if "pre-built binary dependencies" are omitted with mini, what are they being substituted with? Because their functionality must somehow be replaced.

They are not replaced. The feature that needs them will just not work or crash. These are mostly pre-built native binaries of libarchive, 7zip, and libmagic. There is a pale fallback for libmagic though, but that's not enough to get things working.

These are instead installed separately
How?
pip install each one by hand or use scancode-toolkit-mini[full] instead.

or with the extra_requires scancode-toolkit-mini[full]
This is even more confusing: What's the difference between installing scancode-toolkit-mini[full] and scancode-toolkit in the first place?

There should not be much differences at all. In general scancode-toolkit-mini is not what you want unless you know that's what you need and do function-level calls. An example is when it is used in some debian tools to detect copyright one file at a time.

In particular, I do not think that scancode-toolkit-mini is a solution to run on ARM. It may be a stopgap to get something running but is unlikely to be a satisfying solution that has the same features. Instead, what's needed is to port and build the native binaries for ARM too.

[sschuberth]

Thanks for the explanation @pombredanne, and in particular for clarifying on:

In particular, I do not think that scancode-toolkit-mini is a solution to run on ARM.

That's indeed the underlying question in oss-review-toolkit/ort#7097.

Do you think it makes sense to clarify this also in setup-mini.cfg directly?