How to use Transform with a CSV file

[djl197]

My current automation produces a report with the following columns
containing_pkg,pkg_type,pkg_name,pkg_version,pkg_homepage,pkg_licenses

Row0 contains these headings then the following rows contain the data
installer.tar.gz,rpm,apr-util,1.5.2-6.el7,http://apr.apache.org/,ASL 2.0 ++

I tried to apply the transform with something like this:

column_renamings:
    pkg_name : about_resource
    pkg_name : name
    pkg_version : version
    pkg_homepage : homepage_url
    pkg_license : license_expression
column_filters:
    - about_resource
    - name
    - version
    - homepage_url
    - license_expression

Where I believe the filters are applied to the renamed columns.
However I am getting these errors:

CRITICAL: Row 0 is missing required values for columns: about_resource
CRITICAL: Row 1 is missing required values for columns: about_resource
CRITICAL: Row 2 is missing required values for columns: about_resource
CRITICAL: Row 3 is missing required values for columns: about_resource

I suspect user error - but any help appreciated :-)

[chinyeungli]

@djl197
The problem here is the code load the config file in an ordered dictionary list with the original name as a key. Therefore, it will not work if renaming for the same reference as the "old" key's value will be replaced by the "new" key's value. In this case, the first pkg_name: about_resource will be replaced by pkg_name: name and therefore prompt error as the about_resource is no longer in the dictionary list for renaming.

I agree this is a bug and should not restrict not to use the same column for key renaming. (#428)

In the meantime, a work around would be to duplicate the pkg_name column and rename to let say pkg_path and then replace the config file to something like this:

column_renamings:
    pkg_path : about_resource
    pkg_name : name
    pkg_version : version
    pkg_homepage : homepage_url
    pkg_licenses : license_expression
column_filters:
    - about_resource
    - name
    - version
    - homepage_url
    - license_expression

(Note that I have updated the pkg_license from pkg_licenses as your input example use pkg_licenses)

[djl197]

Excellent - thats moved me on nicely - thankyou

[djl197]

I managed to create a CSV file so that I could run:

about gen modified.csv about_files

I then tried to run
about attrib about_files/ attribution.html

But sadly my current report (used as input) provides SPDX license expressions - and from reading I dont think these are supported yet (needs to be DejaCode expressions).

Thankyou for the help - will think about next steps.

[chinyeungli]

Thanks for trying it out. Your input are valuable.
Re the SPDX, we already have ticket for that: #413 (Thou this is old, perhaps it's time to raise the priority)

[pombredanne]

@djl197 note that ScanCode can detect and normalize SPDX license expression to use the Scancode license keys that will then be recognized by AboutCode TK ... so that could be part of solution

[chinyeungli]

The transform issue was described (again) and fixed in #428
The SPDX license issue will be handled in #413
Closing this now.