Scanning package for keyword

[ummer9987]

Description

Is it possible to trigger the scancode for searching a keyword in the packages. If user gives an input keyword is it possible to search that particular keyword in the packages.

Current version 3.0.2

[ummer9987]

@pombredanne could you please help me on this

[pombredanne]

@ummer9987 Thank you for this
@ChinnuJose has also a ticket in #1755

I suggest we have a chat online on Gitter on the details together to hammer a design.
The chat room is at https://gitter.im/aboutcode-org/discuss

[ummer9987]

@pombredanne We are from same team. How will we able to implement this keyword search?

[pombredanne]

@ummer9987 @ChinnuJose if you are from the same team, let's keep only one ticket.

Please work together to close one of the two tickets.

The best place to implement this is IMHO to model after code in the cluecode module. The email and url scans are essentially regex string searches.

That said, there are many design questions pending.

What can be searched?:

• plain strings?
• globs?
• regex?
• would upper/lower case matter?

And then what could be the user experience:

• do we allow to search in binaries or only in plain text files?
• what is the data structure returned and does it include the line numbers?
• is there a configurable limit to the number of returned values (this is something done for email urls)?
• what is the name of new command option(s)?
• can multiple strings be searched at once?

Also this new "scan" could be implemented as an external plugin, but I think it makes ore sense to have it part of the core scancode.

[ChinnuJose]

What can be searched?:

plain strings? - Yes
globs? - Not Mandatory
regex? - Could be
would upper/lower case matter? - No .

And then what could be the user experience: IPR & Legal team can evaluate the package more efficiently and check in detail if specific keyword exist.

do we allow to search in binaries or only in plain text files? - Both
what is the data structure returned and does it include the line numbers? - Yes/No (1/0) . It will be good if we can also get the Line numbers ,no of occurrence ,file
is there a configurable limit to the number of returned values (this is something done for email urls)? : No
what is the name of new command option(s)? : Keyword-search ?
can multiple strings be searched at once? : Yes
Also this new "scan" could be implemented as an external plugin, but I think it makes ore sense to have it part of the core scancode.
It will be good if it is inbuilt, as i think it will be useful for others, without much need to plugin configuration. But how much time will it take to create such an Plugin and then to make the release.

Also, if we can do it externally then how we can integrate this with Scancode?

[pombredanne]

@AyanSinhaMahapatra ping ... see the details above

[ummer9987]

@pombr We were trying to add the sample plugin in https://github.com/nexB/scancode-toolkit/wiki/How-To:-Add-a-post-scan-plugin. We followed all the steps mentioned in the page, but we are getting the below error.
Could not find a version that satisfies the requirement scancode-toolkit-3.0.2 (from scancode-hello==1.0.0) (from versions: )
No matching distribution found for scancode-toolkit-3.0.2 (from scancode-hello==1.0.0)

Could you please help us on this

[ummer9987]

@pombredanne We have managed to install the sample plugin successfully. But after installation we triggered the scan but it is showing Error: no such option: --hello.

Could you please help us on this

[steven-esser]

@ummer9987 Make sure you are in the scancode-toolkit virtualenv before installing the hello sample plugins.

[ummer9987]

@MaJuRG @pombredanne The sample plugin works with one change.
Could you please let me know whether you were able to develop the plugin with the requirements provided above.

[pombredanne]

@ummer9987 @AyanSinhaMahapatra will be working on this at the same time as he writes a tutorial to write plugins. So this will take some time, unless you want to give it a go before that separately?

[ummer9987]

@pombredanne @AyanSinhaMahapatra We are also working on that, will share the updates.

[ummer9987]

@pombredanne We have created keyword plugin using the email plugin code. But when we try to trigger a scan using the --keyword option it is showing no such keyword error. If we avoid that option am getting another error
ERROR: failed to load plugin: scan:search:

Could you please help me on this