Migrate the ProjectError model to a more global ProjectMessage #338

[tdruez]

Those Project issues are not particularly useful and there's often too much of those.
It has the bad effect of hiding more important errors that happened during pipeline runs.

@pombredanne any good reasons to keep creating those?

[pombredanne]

• Not creating these as packages may mean we skip on useful things.
• The error messages are problematic.
• IMHO we should have something that would be some way to provide a warning: here is a Windows executable and we do not have full information on its origin and license. You should look into this

[tdruez]

Not creating these as packages may mean we skip on useful things.

They cannot be created as packages (only resources) at the DB level since those are missing mandatory fields...

[pombredanne]

Packages that are incomplete can or cannot be created (may be they could) when they are missing some key fields BUT in all cases these are warnings and not errors. I suggest these updates

• rename the ProjectError model to ProjectMessage
• add a "severity" field to each error as a choice of "error", "warning", "info".
• the packages that are missing some key fields should de tagged as "warning" rather than error.
• we could display the counters of info/warning/messages rather than only errors.
• later, we could have a way to resolve errors with a new "resolution" field that would store a resolution note when the error is resolved. For instance this kind of error:

ERROR: for scanner: licenses: ERROR: Processing interrupted: timeout after 120 seconds. 	
id: 21457001
md5: 43e47161986663a8d308c08907c21931
name: TestListSkus
path: azure-sdk-for-net-64c6c3b957bd22fb0664bb3b7500c046aaa87ddc.zip-extract/azure-sdk-for-net-64c6c3b957bd22fb0664bb3b7500c046aaa87ddc/sdk/compute/Microsoft.Azure.Management.Compute/tests/SessionRecords/ListSkuTests/TestListSkus.json

could be resolved automatically as "not important" as this is a large (15MB) JSON data files that times out, adn is further a test file.

Also it would be useful to:

• keep a link to the corresponding Resource(s) or Package to help with the investigation.
• always create these packages even if not complete or missing some key information

There also bugs and improvement issues to have fewer errors and messages:

• Empty package-lock.json should not return packages scancode-toolkit#2732
• Return a name for winexe, cab, jar and related packages that do not have structured metadata scancode-toolkit#2733
• Infer version from filenames (in particular package archives) scancode-toolkit#2734