Filter out/ignore copyrights by a specific author/holder

[nakami]

I want to ignore copyright statements by a specific author.
However, there are two sections in scancode-toolkit/src/cluecode/copyrights.py which might adress my problem.

there is the "patterns" array line 97-257, but also "junk" line 627-653.
which will do the job the best?

also, a bit more documentation on that matter would be great.

[pombredanne]

@nakami sorry for not replying earlier to this ticket!
I think the best way to implement this would be to have some settings (possibly as a command line option and/or a .scancode.yml settings file where you would list exclusion patterns that if matched would exclude the matching detected copyright

So the copyright would be always detected, but filtered out if it matches an exclusion pattern
As a refinement the patterns could be based on the copyright holder only, but that is likely too much for a start.

It might much better than changing the copyright grammar.

Let me add some docs on the way the copyright detection works to the wiki

[pombredanne]

vaguely related, @jdaguil pushed a new feature to a branch https://github.com/nexB/scancode-toolkit/tree/214-license-policy : this allows to define a policy on licenses.
In a similar way I could think of a feature where a list of authors is "white listed" and either filtered out of considered a meeting a policy expectation.

[sschuberth]

I'm also interested in this feature. Do you think it would make sense to build on the 214-license-policy branch and allow to add regex patterns to .scancode.yml that filter out specific Copyright findings?

[pombredanne]

@sschuberth you wrote:

Do you think it would make sense to build on the 214-license-policy branch and allow to add regex patterns to .scancode.yml that filter out specific Copyright findings?

it would definitely make sense! and this could overtime evolve to offer provide other scan filtering and failure conditions capabilities.

[haikoschol]

@pombredanne I've been looking into this issue and went down the rabbit hole of related issues, feature branches and proposals. Now I'm trying to figure out how I can address this specific issue while at the same time avoid

1. addressing all related issues and the architecture/refactoring work that entails
2. putting in a quick hack that makes 1. harder down the line

At the moment I'm thinking a plugin would be the simplest and cleanest way to filter the results. However, the question then would be how that plugin gets the exclusion patterns to apply. Either by adding a way to pass command line parameters to plugins (#748 and #787) or by reading them from a config file (#520 and the 214-license-policy branch).

What are your thoughts on this? And @yasharmaster, since you are working on #787, what's your take?

[pombredanne]

@haikoschol you wrote:

At the moment I'm thinking a plugin would be the simplest and cleanest way to filter the results.

that's the way.

However, the question then would be how that plugin gets the exclusion patterns to apply.

the key will be to have #787 for you to do something clean
@yashdsaraf 'sup on this front?

[...] or by reading them from a config file (#520 and the 214-license-policy branch).

I think that should come for free and optionally with a generalized config file such that this could store any number of repetitive options.

Actually @yashdsaraf is on #787 ... but @yasharmaster your contribution would always be welcomed too!

[haikoschol]

Ah, sorry about that @yasharmaster and @yashdsaraf. Just took the completion result and didn't double check it.
@pombredanne So eventually you want to have command line parameters for plugins as well as a config file that includes options for plugins, right? Ideally the plugins just receive parameters via __init__() and the cli code around them figures out where they come from and which ones have precedence.

[pombredanne]

@haikoschol exactly 👍
Now you are raising an interesting issue: precedence in plugins... hum. That may need to be specified indeed. Do you see a use case where several post-scans plugins would require to run in a certain order? e.g. where one would depend on the other?

[haikoschol]

@pombredanne Actually you raised that issue, because I was thinking about precedence among command line options and a config file. :)
I think the result of a scan should be independent of the order in which plugins are executed.

Regarding the other precendence issue, I'd say these are the options:

1. merge all options giving precedence to one kind over the other (I find cmdline over config file intuitive)
2. when both are given, completely ignore one (again, I'd expect cmdline to override config file)
3. only allow settings in the config file that don't exist as cmdline options yet
4. remove all but the most essential cmdline options and move all others to the config file

I'd pick option 4, if you are ok with breaking backwards compatibility (maybe in 3.0?). Also considering that there already are tickets related to simplifying/refactoring command line options. However, a monolithic config file can also get out of hand easily. For plugin options we would probably need namespacing. And then there's the question which ones are "essential", of course.

[pombredanne]

@haikoschol thank you for a great write up.

I am split between exposing all options in the CLI or only some options. The traditional pattern is to have everything in the CLI but that can become hairy but a config file can be hairy too. Let me sleep over this. @jpopelka what would be you take?