Make transform work - #383
Closed
pombredanne wants to merge 69 commits into
Closed
Conversation
pombredanne
force-pushed
the
361-make-transform-work
branch
from
December 13, 2018 18:24
e7dc08f to
703e4f4
Compare
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
- use simpler attr for data models definition: this is simpler and cleaner. - update all commands and tests accordingly - introduce a proper License objects to hold the "licenses" details - create new fetch-licenses subcommand to retrieve License objects through API calls. - store reference and API-fetched License as a pair of xxx.yml data file and xxx.LICENSE text file - move all inventory-related ode to inv.py - update built-in templates to use the new About and License objects Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
This is in the inv/check commande Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
This guarantees that the same attribution will be generated on all OSes Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
This was not enabled and therefore HTML was not esacped correctly. Also streamline the default template whitespaces. Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
This ensure that we will use the /licenses/ end point even if not provided explicitly. Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Python, EPL and AGPL are common enough Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
This is: about_resource, name, version Also remove duplicated field definition code Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Instead use the about_resource<.ABOUT> Use about_file_path if an only if provided. Alos remove --quiet option rom Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Instead use the about_resource<.ABOUT> Use about_file_path if an only if provided. Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
* Add a new "reformat" command to reformat an existing tree of ABOUT files to use the new cleaner YAML format and the new standard fields ordering. This helps compare ABOUT file generated with 3.3.0 with v4.x ABOUT files for verification purpose. This can also help normalize existing ABOUT files in general * Add a new "--legacy-placement" option to the gen command. With this option, the placement of ABOUT files is inside a documented directory as opposed to be side-by-side, which is the way in 3.3.0. This helps to compare generated trees of ABOUT files between 3.3.0 and 4.x for verification * Improve handling of paths (about resource vs. about file path) in gen.py Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
This makes it easier to write templates Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Also rename legacy vartext to vartext_dict to backward compat Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Member
Author
|
At this stage, the features are all tested and working and at least at parity with the v3.3.0. This is ready for merging |
The new --unique-fields option is a list of field names separated by commas. When provided the ABOUT entries provided to a custom attribution template is dedulicated using the values of these fields considered together a unique key. When several ABOUT entries have the same unique key, only the first one is kept. Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
* Also use enterprise.dejacode.com to check connectivity * Use .format for string templates Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
* Add two new template filters to sort and control unicity of list items using the values of multiple attributes. The standard sort and unique Jinja2 filters only support a single attribute value. The new multi_sort and unique_together filters take an "attributes" list of attribute names as an argument and use the values of all these attributes to sort or deduplicate a list or iterable. * Remove --unique-fields command line option from attrib command Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
This helps with a backport to the v3 branch Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
This helps with a backport to the v3 branch Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Contributor
|
I am closing this as we already have the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This makes the "transform" CSV to CSV command work for #361 but also replaces entirely the internal custom field validation by the
attrlibrary and works on other tickets:Clarify or remove "about_file_path" attribute #364 the about_file_path is no longer an attribute: this is only something provided in the inventory used by the gen command and is not stored: it is only used to craft the location where the ABOUT file will be created.
Add support for special characters in .ABOUT filenames #378 removes limit on characters used in .ABOUT file names
The
namefield should not be required #379 the name attribute is no longer requiredDrop long/deep path extra support code #382 deep path support on Windows requires using Python 3.6+ per https://docs.python.org/3/using/windows.html#removing-the-max-path-limitation
multiple licenses support for json output #344 proper support for Licenses list
Make
--fetch-licenseits own subcommand instead of a gen option #308 add new fetch-licenses sub command