adding new license

[friczb]

Jun 3, 8:07 AM
Hi,
today @pierrelapointe showed me a quick preview of ScanCode. It looks really promising.
I have one question what he suggested the write here.

Can you add/define a new license in ScanCode?
Can you add the regexp for the this new license?
Can you filter for this new license on the summary page?
Can it be synchronized with DejaCode license library?

For referrence I include here the way how Fossology works. It's really a pain.
http://www.fossology.org/projects/fossology/wiki/Nomos#How-to-Add-a-New-License-Signature

This can be really a good USP.
Kind regards,
Béla

[pombredanne]

Can you add/define a new license in ScanCode?

Yes, just add a xxx.LICENSE file with the license text and a small xxx.yml file with the license "metadata" in YAML format in the src/licensedcode/data/license directory and you are done.

Select xxx to be a new unique license key for your new license. Unique means: it does not already exist in the data/license directory.

Use the many existing .yml files as examples as in https://github.com/nexB/scancode-toolkit/tree/master/src/licensedcode/data/licenses

By adding only this two simple files, your new license can be detected immediately.

For more comprehensive instructions see:
https://github.com/nexB/scancode-toolkit/blob/master/README.rst#how-to-add-a-new-license-for-detection

Can you add a regexp to detect this new license?

First there is no need to add a regexp to detect a license. The license text is used for detection first.
Then for improved detection, we do not use regexp but simpler license rules that are plain text examples of a license mention in addition to the full license text.

Just add a yyy.RULE file with the license example and a yyy.yml file with the license key that should be detected in a small rule "metadata" in YAML format in the src/licensedcode/data/rule and you are done. You need to have a corresponding license key created first of course, see above.

Select yyy to be a unique name prefix for your new rule. By convention we like to use the detected license keys in the file name.

Use the many existing .yml files as examples as in https://github.com/nexB/scancode-toolkit/tree/master/src/licensedcode/data/rules

By adding only this two simple files, your new rule is available immediately for improved license detection.

For more comprehensive instructions including pattern-like rules using "gaps" in text see:
https://github.com/nexB/scancode-toolkit/blob/master/README.rst#how-to-add-a-new-license-detection-rule

Also a test can be added for verification using the same approach: a file to test and a .yml containing the license key that are expected to be found when scanning.

See https://github.com/nexB/scancode-toolkit/tree/master/tests/licensedcode/data/licenses

Can you filter for this new license on the summary page?

Yes, when you create an html-app output with scancode --format=html-app .... you can search, sort and filter based on the license name or key or category or anything else, using the search box.

You can see that search box here: https://github.com/nexB/scancode-toolkit/blob/master/samples/screenshot.png

Can licenses be synchronized with the DejaCode license library?

The license keys are the same that are used in DejaCode. They are kept in sync by hand in the short term.
There is also a ticket to automate that sync with DejaCode and possibly other sources. See #41

[pombredanne]

And of course you can keep these new licenses and rules private or contribute them back to ScanCode so they become part of the standard data set.

[pombredanne]

And checking that fossology link, you can see that there is no need to recompile, no database to change or system to restart.
A simple license yml file looks like this:
https://github.com/nexB/scancode-toolkit/blob/master/src/licensedcode/data/licenses/aes-128-3.0.yml

key: aes-128-3.0
short_name: AES-128 3.0 License
name: AES-128 v3.0 License
category: Public Domain

And a simple rule (here to detect a license based on a URL) looks like this:
https://github.com/nexB/scancode-toolkit/blob/master/src/licensedcode/data/rules/agpl-3.0_9.yml
and this https://github.com/nexB/scancode-toolkit/blob/master/src/licensedcode/data/rules/agpl-3.0_9.RULE

[pombredanne]

@friczb was the information I provided above useful?

[friczb]

Thank you, @pombredanne ! This was really useful! And it is clear now that your solution provides more (and easier) than Fossology.
Have a nice day!
B

[pombredanne]

@friczb Thank you for the feedback and questions
I added these to the FAQ:
https://github.com/nexB/scancode-toolkit/wiki/FAQ