TBD.
This is a major version with API-breaking behavious changes in the reosurce module.
The Resource has no rid (resource id) and no pid (parent id). Instead we now use internally a simpler mapping of {path: Resource} object. As a result the iteration on a Codebase is faster but this requires more memory.
The Codebase and VirtualCodebase accepts a new "paths" argument that is list of paths. When provided, the Codebase will only contain Resources with these paths and no other resources. This handy to create a Codebase with only a subset of paths of interest. When we create a Codebase or VirtualCodebase with paths, we also always create any internediate directories. So if you ask for a path of "root/dir/file", we create three resources: "root", "root/dir" and "root/dir/file". We accumulate codebase errors if the paths does not exists in the Codebase or VirtualCodebase. The paths must start with the root path segment and must be POSIX paths.
When you create a VirtualCodebase with multiple scans, we now prefix each scan path with a codebase-1/, codebase-2/, etc. directory in addition to the "virtual_root" shared root directory. Otherwise files data was overwritten and inconsistent when each location "files" were sharing leading path segments. So if you provide to JSON inputs with that each contain the path "root/dir/file", the VirtualCodebase will contain these paths:
- "virtual_root/codebase-1/root/dir/file"
- "virtual_root/codebase-2/root/dir/file"
It is otherwise practically impossble to correctly merge file data from multiple codebases reliably, so adding this prefix ensures that we are doing the right thing
The Resource.path now never contains leading or trailing slash. We also normalize the path everywhere. In particular this behaviour is visible when you create a Codebase with a "full_root" argument. Previously, the paths of a "full_root" Codebase were prefixed with a slash "/".
When you create a VirtualCodebase with more than one Resource, we now recreate the directory tree for any intermediary directory used in a path that is otherwise missing from files path list. In particular this behaviour changed when you create a VirtualCodebase from a pervious Codebase created with a "full_root" argument. Previously, the missing paths of a "full_root" Codebase were kept unchanged. Noet that the VirtualCodebase has always ignored the "full_root" argument.
The Codebase and VirtualCodebase are now iterable. Iterating on a codebase is the same as a top-down walk.
Other changes:
- Remove Python upper version limit.
- Merge latest skeleton
- fileutils.parent_directory() now accepts a "with_trail" argument. The returned directory has a trailing path separator unless with_trail is False. The default is True and the default behavious is unchanged.
- Relax dependencies version requirements by removing upper bounds.
- Use latest skeleton.
- Minor improved utilities
- More robust handling of Codebase with a single Resource
- Remove usage of deprecated
click.get_terminal_size().
- Add
warningfield tocommoncode.Codebaseheaders. - Add new functions
get_jar_nv()andget_nupkg_nv()that accepts a filename of a JAR or nupkg and return a name/version tuple extracted using multiple heuristics.
- Switch back from clamver to semver.
- Adopt latest skeleton. The default virtualenv directory is now venv and no longer tmp
- Fix issue with Click progressbar API #23 that prohibited to use all supported Click versions. Since Click is widely used that was a frequent source of installation conflicts.
- Add an attribute to the header for scancode output format versioning. This is for aboutcode-org/scancode-toolkit#2653
- Ensure that the progressbar displays a counter correctly. This is a fix for aboutcode-org/scancode-toolkit#2583
- Add preserve_spaces argument in commoncode.paths.portable_filename. This argument will prevent the replacement of spaces in filenames.
- Do not fail if a Codebase does not have a common shared root #23
- Consider all Resource attributes when building a VirtualCodebase #23
- Do not ignore by default sccs and rcs dirs aboutcode-org/scancode-toolkit#1422
- Do not fail if a Codebase file.size is None and not zero
- Bump pinned dependencies including pkg:pypi/urllib3 for CVE-2021-33503
- Fix click-related bug aboutcode-org/scancode-toolkit#2529
- Add tests to run on the latest of every dependency
- Add new function to find a command or shared object file in the PATH (e.g. in environment variables). See commoncode.command.find_in_path()
- Add new simplified the commoncode.command.execute() function.
- Add support for Python 3.10
- Update tests to cope with Python 3.6 bug https://bugs.python.org/issue26919
- Adopt latest skeleton with configure scripts updates
Breaking API changes:
- commoncode.command.load_shared_library() now ignores the lib_dir argument
- commoncode.command.execute2() is deprecated and ignores the lib_dir argument it is replaced by commoncode.command.execute()
- In commoncode.testcase get_test_loc() "exists" argument has been renamed to "must_exist". It has also been added to FileDrivenTesting.get_test_loc() method.
- Add new function to get a Resource path stripped from its root path segment
- Improve error reporting when oding missing DLLs
- Clean config and improve basic documentation
- Update dependencies
- Add Azure Pipelines CI support
- Drop Python 2 support
- Update license
- Add support for both python 2 + 3
- Add CI support for python 2 + 3
- Minimal fixes needed for proper release
- Update to PEP 517/518 development practices
- Add some minimal documentation
- Initial release.