New aboutcode-toolkit-3.4.0.pre1 transform option is missing --help-format

[DennisClark]

I could not get the new aboutcode-toolkit-3.4.0.pre1 transform option to work on an existing inventory .csv file using an existing confg file, so I tried exploring the help to see what needed to be done, thinking it might be a config file problem.

I ran ./about transform -h and learned that there ought to be a --help-format option.

So I ran ./about transform --help-format and got a bunch of error messages. Please advise or fix.

(aboutcode-toolkit-3.4.0.pre1) dennis-clarks-macbook-pro:aboutcode-toolkit-3.4.0.pre1 dclark1330$ ./about transform -h
Usage: about transform [OPTIONS] LOCATION OUTPUT

  Transform the CSV file at LOCATION by applying renamings, filters and
  checks and write a new CSV to OUTPUT.

  LOCATION: Path to a CSV file.

  OUTPUT: Path to CSV inventory file to create.

Options:
  -c, --configuration FILE  Path to an optional YAML configuration file. See
                            --help-format for format help.
  --help-format             Show configuration file format help and exit.
  -q, --quiet               Do not print error or warning messages.
  --verbose                 Show all error and warning messages.
  -h, --help                Show this message and exit.
(aboutcode-toolkit-3.4.0.pre1) dennis-clarks-macbook-pro:aboutcode-toolkit-3.4.0.pre1 dclark1330$ ./about transform --help-format
Traceback (most recent call last):
  File "/Users/dclark1330/aboutcode-toolkit-3.4.0.pre1/bin/about", line 11, in <module>
    load_entry_point('aboutcode-toolkit', 'console_scripts', 'about')()
  File "/Users/dclark1330/aboutcode-toolkit-3.4.0.pre1/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/Users/dclark1330/aboutcode-toolkit-3.4.0.pre1/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/Users/dclark1330/aboutcode-toolkit-3.4.0.pre1/lib/python3.6/site-packages/click/core.py", line 1064, in invoke
    sub_ctx = cmd.make_context(cmd_name, args, parent=ctx)
  File "/Users/dclark1330/aboutcode-toolkit-3.4.0.pre1/lib/python3.6/site-packages/click/core.py", line 621, in make_context
    self.parse_args(ctx, args)
  File "/Users/dclark1330/aboutcode-toolkit-3.4.0.pre1/lib/python3.6/site-packages/click/core.py", line 880, in parse_args
    value, args = param.handle_parse_result(ctx, opts, args)
  File "/Users/dclark1330/aboutcode-toolkit-3.4.0.pre1/lib/python3.6/site-packages/click/core.py", line 1404, in handle_parse_result
    self.callback, ctx, self, value)
  File "/Users/dclark1330/aboutcode-toolkit-3.4.0.pre1/lib/python3.6/site-packages/click/core.py", line 78, in invoke_param_callback
    return callback(ctx, param, value)
  File "/Users/dclark1330/aboutcode-toolkit-3.4.0.pre1/src/attributecode/cmd.py", line 399, in print_config_help
    from attributecode.transform import tranformer_config_help
  File "/Users/dclark1330/aboutcode-toolkit-3.4.0.pre1/src/attributecode/transform.py", line 24, in <module>
    import attr
ModuleNotFoundError: No module named 'attr'
(aboutcode-toolkit-3.4.0.pre1) dennis-clarks-macbook-pro:aboutcode-toolkit-3.4.0.pre1 dclark1330$ 

[pombredanne]

The code in the new branch https://github.com/nexB/aboutcode-toolkit/tree/361-make-transform-work and #383 should be working now.
You can try it by downloading https://github.com/nexB/aboutcode-toolkit/archive/361-make-transform-work.tar.gz

[DennisClark]

Suggested branch downloaded. And the help on transform is working now:

./about transform --help-format

A transform configuration file is used to describe which transformations and
validations to apply to a source CSV file. This is a simple text file using YAML
format, using the same format as an .ABOUT file.

The attributes that can be set in a configuration file are:

* column_renamings:
An optional map of source CSV column name to target CSV new column name that
is used to rename CSV columns.

For instance with this configuration the columns "Directory/Location" will be
renamed to "about_resource" and "foo" to "bar":
    column_renamings:
        'Directory/Location': about_resource
        foo: bar

The renaming is always applied first before other transforms and checks. All
other column names referenced below are these that exist AFTER the renamings
have been applied to the existing column names.

* required_columns:
An optional list of required column names that must have a value, beyond the
standard columns names. If a source CSV does not have such a column or a row is
missing a value for a required column, an error is reported.

For instance with this configuration an error will be reported if the columns
"name" and "version" are missing or if any row does not have a value set for
these columns:
    required_columns:
        - name
        - version

* column_filters:
An optional list of column names that should be kept in the transformed CSV. If
this list is provided, all the columns from the source CSV that should be kept
in the target CSV must be listed be even if they are standard or required
columns. If this list is not provided, all source CSV columns are kept in the
transformed target CSV.

For instance with this configuration the target CSV will only contains the "name"
and "version" columns and no other column:
    column_filters:
        - name
        - version

However, since this is rather new stuff, and rather different from the old mapping config style, it might take me a little while to setup a transform configuration file for my test data to determine if the help text matches code reality. I shall report the results as soon as possible.

[pombredanne]

This is now all fixed in the https://github.com/nexB/aboutcode-toolkit/tree/361-make-transform-work
Note that column_filters has been renamed to kept_columns.