Filter Ignorable Copyrights #372 - #2436
Closed
adityaviki wants to merge 1728 commits into
Closed
adityaviki wants to merge 1728 commits into
adityaviki wants to merge 1728 commits into
Conversation
Contributor
Author
|
@pombredanne let me know if I need to make any improvements to the code. I'll write the the tests for it afterwards. |
Contributor
|
Can you please edit the top comment and linked issue no along with your name! This makes easy to understand in that way:) |
pombredanne
requested changes
Mar 24, 2021
pombredanne
left a comment
Member
There was a problem hiding this comment.
Thank you ++
This is looking great! 🙇
It is missing one thing though before I can merge: some test.
Do you think you can draft one?
Contributor
Author
Yes of course :) |
adityaviki
added a commit
to adityaviki/scancode-toolkit
that referenced
this pull request
Mar 30, 2021
Signed-off-by: adityaviki <adityaviki01@gmail.com>
Contributor
Author
|
I didn't mean to close the PR or delete the branch. I wasn't paying attention to where I was clicking. :/ |
pombredanne
requested changes
Mar 30, 2021
Member
that's OK :) |
pombredanne
reviewed
Mar 30, 2021
pombredanne
approved these changes
Mar 30, 2021
Contributor
Author
|
@pombredanne I don't know how to solve the failing check. |
Contributor
DCO check is falling because of this 55cea26 It has co-authour name but signed-off is missing. So what you can do is squash these changes to the original commit of your and add sign-off or you may ask @pombredanne to squash all the commits into one. |
Contributor
Author
|
@itssingh It's still not clear how to solve the issue. |
…tfiles Add CI for Docs and ABOUT files
This is for PR aboutcode-org#1864 and aboutcode-org#1759 Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
This was triggering a FutureWarning Reported-by: Thorsten Godau @tgodau Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Various GPL 3.0 licenses Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Avoid ambiguous license detection as MIT instead of Apache Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
This is for aboutcode-org#1759 as a follow up to PR aboutcode-org#1864 Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Adopt SPDX v2.2 and fix SPDX TV correctness
And spell spurious with one r, not two. Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
This support new ways to combine multiple license matches together, and distinguish primary and secondary licenses. Most concepts have been drawn from early work on Debian copyright detection improvements. Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
…-improvements Add new licenses and new detection rules
Rather than using a function argument Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
This is a batch of misc. license detection rules and a new license. These have been mostly found thanks to the upcoming unknown license detection. Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
This makes it available in the CLI Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
This is cleaner and more composable than old-style interpolation Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Add split_weak_matches() function to pre-filter weak unknown matches. Make unknown matches eligible for filter_spurious_matches() and lower minimum density to 0.6=5 for longer matches. Move the call for filter_spurious_matches() earlier in the refine pipeline. Add new filter_invalid_contained_unknown_matches() function to discard unknown matches found inside the matched queryregion of larger regular matches Extract get_full_qspan_matched_text() function from get_full_matched_text() for improve reusability. This is designed to be called when crafting new rules absed on a match (which is what is done with unknown matches). Use format for matched license text highlight This is cleaner and more composable than old-style interpolation. Improve debug tracing of matched texts. Apply other minor refactoring and doc impropvements Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Create unique rule id based on a checksum of the rule content
Also improve key phrases parsing for dnagling {{ {{ braces.
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Create proper synthethic Rule and LicenseMatch on match and return a match or None. Include unknown licenses matching as an option to Index.match Add tests Use shorter ngrams of length 6 rather than 7 for better sensitivity This is balanced by the addition of filters: - Filter weak unknown matches at match time - Filter out several weak unknown ngrams at index time Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Note that this is NOT YET returned in the API and outputs Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
…tection Detect unknown licenses aboutcode-org#1675 Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
In copyright detection results, rename the "value" attribute to proper descriptive names, respectively: copyright, holder and author Signed-off-by: akugarg <akanksha.garg2k@gmail.com> Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
This second commit updates the tests and code that did not exist yet at the time of the initial patch or code that failed to pass tests. In copyright detection results, rename the "value" attribute to proper descriptive names, respectively: copyright, holder and author. Also: - rename "copyright_holder" to "holder" in CSV output - apply minor refactorings Co-authorred-by: akugarg <akanksha.garg2k@gmail.com> Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
We now use a set of Detection objects to streamline how things are handled internally. Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
…t-handling Improve copyright handling aboutcode-org#2350
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Member
|
@adityaviki this will be merged at last in v32.1 ... but it may be partially redundant with pre-existing ScanCode CLI options |
Member
|
https://github.com/nexB/scancode-toolkit/blob/develop/src/cluecode/plugin_filter_clues.py has the feature and predates this PR. Thank you +++, closing. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A post-scan plugin to filter out ignorable-copyrights that are read from license's yaml file.