Generate streamlined analysis workfile (CSV) from SCIO-DB

[mjherzog]

A streamlined CSV workfile will be very useful for SCA planning. The columns we need are listed below by SCTK runtime option using current SCTK CSV output column names. If there are multiple values in a JSON field we want all of the values in one cell ("flattened").

Info:
Resource
type
name
base_name
extension
size
sha1
mime_type
file_type
programming_language

Copyrights:
copyright
copyright_holder
author

Licenses:
license_expression
license__key
license__score
license__category
license__owner

email
url

Packages:
package__type
package__namespace
package__name
package__version
package__primary_language
package__description
package__release_date
package__homepage_url
package__download_url
package__size
package__sha1
package__vcs_url
package__copyright
package__license_expression
package__declared_license
package__notice_text

[steven-esser]

@JonoYang has written this utility: https://github.com/nexB/spats/blob/develop/src/spats/scanpipe_results_to_xlsx.py that "flattens" the data how we normally expect it (\n characters separating multiple values) and supports package data.

@mjherzog I believe we want to support package data as well? This is on our normal work file alongside regular scancode detections.

[mjherzog]

@MaJuRG My mistake to forget about packages. I updated the main text above with selected package fields.

[mjherzog]

The list of fields requested is my take on what is most useful for planning and simpler analysis tasks. My working assumption is that there will be other CSV workfile output variations.

[tdruez]

Closing as the existing XLSX output should be used instead.