Collect scanned license texts

[pombredanne]

Together with a license scan, it would be useful to collect the matched license texts.
We have all the data in the scans, it is just a matter of extraction.
A script could do it, but a better solution would be a command line option.

[balusarakesh]

It might be a good idea to show the matched text in the scancode html-app too.

[tags07]

+1 I'd like this feature too.
a. Would be nice to collect all copyright+license
b. filter unique copyright+license to get a minimal set of attributions in the source

[pombredanne]

@tags07
For a. copyrights are already collected, together with holders. A holder is the way to get a minimal set IMHO

The license texts require to effectively get the text. We know exactly in a scan where there was a match and there is even a function in a branch to collect the texts. https://github.com/nexB/scancode-toolkit/blob/57113ee4231789803f0df10c0658fd683b810ac2/src/licensedcode/query.py#L203
This is not yet exposed in the results. Doing it is the way to include the texts in the results.
Would you envision the actual JSON results to contain the full texts?

For b. in general this is the hard part: summarization....
It all depends on which type of scans you do: if for instance this is repo by repo, then that should work nicely. On the other hand if this is for a whole or large codebase, this becomes hairy.
What's your main use case?

[tags07]

@pombredanne,
JSON would be good. But even a simple text/csv export would suffice for example:

 <file1>
 <copyright statement>
 <License Text>
==================
<file2>
<copyright statement>
<License Text>

==================
and so on...

Regarding b. I agree with you. The use case will be to summarize the license text for each repo. If a repo has 100 files with each file containing same BSD header, it would be nice to summarize the license of that repo and produce a single BSD text.
This could even help generate a starter NOTICE file that we could manually scrub to produce the final version. Would be better than manual copy+paste

[pombredanne]

So the next step will be to collect full notices.
The latests commits are a first step toward a comprehensive "notice" capture. A full notice capture would apply a few heuristics and combine detected copyrights, urls, emails and license when closely related and co-located so these are accurate.