For Maven declared_license normalization, also consider all text at once as well as POM XML comments

[pombredanne]

Description

The licenses tags in a Maven POM are normalized using license tags in the XML but we ignore XML-wide comments. It si common in POMs to have a license in an XML top-level comment. Licenses in POM XML comments would be caught by the plain text detection alright but we want then in the package licenses instead.

We should enhance this by:

1. Looking at the text made of lines of (name, url, comments)
2. Considering the XML comments in the package normalization too
   3

[reversi-fun]

That Great!,if the following normalization is included.

• biz.aQute.bndlib.1.43.0.pom.xml
• Detect 2 legal NOTICE: spdx/Apache-2.0, "see swing2swt".
  https://search.maven.org/artifact/biz.aQute/bndlib/1.43.0/jar

    <licenses>
        <license>
         <name>
         All files contained in this JAR are licensed under the Apache
         2.0 license, unless noted differently in their source (see
         swing2swt).
        </name>
      <url>
        http://www.opensource.org/licenses/apache2.0.php
      </url>
      <distribution>
        repo
      </distribution>
     </license>

• org.jdom:jdom:1.1.2. pom.xml
• Detect 1 unique license Name not spdx/Apache-1.1.
  https://search.maven.org/artifact/org.jdom/jdom/1.1.2/jar

<licenses>
  <license>
    <name>Similar to Apache License but with the acknowledgment clause removed</name>
    <url>https://raw.github.com/hunterhacker/jdom/master/LICENSE.txt</url>
   <distribution>repo</distribution>
  </license>
</licenses>

• I found the following aliases: But I was tired.
  https://github.com/reversi-fun/license_doc_similality1/blob/master/config/license_alias.csv
  https://github.com/reversi-fun/license_doc_similality1/blob/master/data/Tree_Howers_works_of_filePattern2License.csv

[pombredanne]

See some attached example of poms with license comments at the top:
pom.xml.txt
log4j-1.2.15.pom.txt

[pombredanne]

See also https://repo1.maven.org/maven2/org/apache/logging/log4j/log4j/2.4/log4j-2.4.pom
@AyanSinhaMahapatra suggested to do this during the package scan

• a. scan for manifest data to get the extracted license
• b. scan also the whole file as plain license detection
  Combine the results of of b. in a. (possibly updating the extract_license_statement based on the matched text of b.) using something like we have for combining license matches in detections.