Compute summary and clarity for EACH package in a codebase

[pombredanne]

Today the summary and license clarity scores are computed for the whole scan. Instead we should compute them for EACH package (and their files). This is possible now that we are returning which file belong to a package,

[pombredanne]

This new feature would serve also aboutcode-org/scancode.io#86 and may be aboutcode-org/scancode.io#631

One of the main usage is that we have cases where the package manifest are missing key metadata such as license or copyright as well as various URLs. Yet this information may exists in a package key files and this would be a mjor improvement to collect this data smartly.

Since we can now properly detect which file belong to a package, we can therefore compute a summary of these files the same way we can compute a codebase summary with the --summary option

This new feature could add a new --package-summary option as a post scan plugin that would depend on the availability or behave based on the availability of the license and copyright scan, as well the --classify option. This should not replace the --summary but should instead share most of the code.

[pombredanne]

See also the related #3290