Skip to content

#99 license detection in binaries fails sometime - #353

Merged
pombredanne merged 26 commits into
developfrom
99-license-in-binary
Nov 4, 2016
Merged

#99 license detection in binaries fails sometime#353
pombredanne merged 26 commits into
developfrom
99-license-in-binary

Conversation

@pombredanne

@pombredanne pombredanne commented Nov 4, 2016

Copy link
Copy Markdown
Member

These commits provide a fox #99 and bring several derived improvements that were needed or peripheral to the fix.

  • revamped extraction of strings from binaries
  • improved license matches merging and filtering
  • new and improved or deleted rules and licenses
  • improved tests and test expectations

pombredanne and others added 26 commits November 3, 2016 17:31
 * moved stlport-2000 and rules to "notices" and added test
 * deleted halpern-wight which is a dupe of stlport-4.5
 * added various rules for mozilla-gc and stlport-related licenses
 * added new historical rules (for stlport-200 detection)

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
 * this is a subset of the strings extracted from libgcj.so.10.0.0
   originally from
http://vault.centos.org/6.4/cr/x86_64/Packages/libgcj-4.4.7-4.el6.i686.rpm
    /usr/lib/libgcj.so.10.0.0

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
 * otherwise too many query runs are created

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
 * use a regex-based approach as in fireeye's floss
 * simplify filtering
 * now supporting UTF-16 strings in Windows PE

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
 * This is no longer needed now that we have a more accurate detection
   Also updated rules and tests accordingly.  

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
 * ensure that we do not merge matches that would result in
   unaligned ispan/qspan
 * keep the densest of two matches overlapping with equal ispans

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
 * Prior to this, strings needed to have at least three unique characters
   which was a mistake.
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
 * remove duplicated SPDX text
 * remove copyright statement

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
 * add new rules and texts (including a general CLA)
 * refine frequent tokens
 * add new disambiguation with a filter for matches that overlap
   multiple matches
 * correct the licensing identifier of rules that were all incorrect
   (and were set to False)

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
 * handle better some trailing words and complex university names
 * handle "Portions Copyright ..."

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
pombredanne merged commit 5aa84c1 into develop Nov 4, 2016
@pombredanne
pombredanne deleted the 99-license-in-binary branch November 4, 2016 22:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants