during pyside6-essentials scan, scancode does not detect LGPL/GPL licenses in files

[yahym]

Description

Please leave a brief description of the bug or feature request:

During pyside6-essentials scan, scancode does not detect LGPL/GPL licenses in files:
Question: Are the LGPL/GPL used in this package not standardized or why are these licenses not detected?

How To Reproduce

Tell us how to reproduce the issue.

rem get PySide6-Essentials from https://files.pythonhosted.org/packages/55/e0/ccbf260a6545460d1da255810320d4f0cc8aee7fe4127eec07cfa6297228/PySide6_Essentials-6.3.0-cp36-abi3-win_amd64.whl

set "suffix=PySide6_essentials"
set "folder=PySide6"
set "file_name=%suffix%"

call scancode.exe ^
    -lcp ^
    -n 5 ^
    --csv %file_name%.csv ^
    --html %file_name%.html ^
    --json %file_name%.json ^
    --license-text ^
    --unknown-licenses ^
    --only-findings ^
    --consolidate ^
    --filter-clues ^
    --is-license-text ^
    --info ^
    --license-text ^
    --license-text-diagnostics ^
    --license-clarity-score ^
    --summary ^
    --classify ^
    --full-root ^
     %folder%

System configuration

For bug reports, it really helps us to know:

• What OS are you running on? (Windows/MacOS/Linux): Windows 10 19042
• What version of scancode-toolkit was used to generate the scan file?: 30.1.0
• What installation method was used to install/run scancode? (pip/source download/other):

Example:
From file d:\PySide6_Essentials-6.3.0-cp36-abi3-win_amd64\PySide6\qml\QtQuick\Controls\Universal\MenuSeparator.qml

/****************************************************************************
**
** Copyright (C) 2017 The Qt Company Ltd.
** Contact: https://www.qt.io/licensing/
**
** This file is part of the Qt Quick Controls 2 module of the Qt Toolkit.
**
** $QT_BEGIN_LICENSE:LGPL$
** Commercial License Usage
** Licensees holding valid commercial Qt licenses may use this file in
** accordance with the commercial license agreement provided with the
** Software or, alternatively, in accordance with the terms contained in
** a written agreement between you and The Qt Company. For licensing terms
** and conditions see https://www.qt.io/terms-conditions. For further
** information use the contact form at https://www.qt.io/contact-us.
**
** GNU Lesser General Public License Usage
** Alternatively, this file may be used under the terms of the GNU Lesser
** General Public License version 3 as published by the Free Software
** Foundation and appearing in the file LICENSE.LGPL3 included in the
** packaging of this file. Please review the following information to
** ensure the GNU Lesser General Public License version 3 requirements
** will be met: https://www.gnu.org/licenses/lgpl-3.0.html.
**
** GNU General Public License Usage
** Alternatively, this file may be used under the terms of the GNU
** General Public License version 2.0 or (at your option) the GNU General
** Public license version 3 or any later version approved by the KDE Free
** Qt Foundation. The licenses are as published by the Free Software
** Foundation and appearing in the file LICENSE.GPL2 and LICENSE.GPL3
** included in the packaging of this file. Please review the following
** information to ensure the GNU General Public License requirements will
** be met: https://www.gnu.org/licenses/gpl-2.0.html and
** https://www.gnu.org/licenses/gpl-3.0.html.
**
** $QT_END_LICENSE$
**
****************************************************************************/

Result (a kind of LGPLv3 and GPLv3 or later license were also expected):

[yahym]

my mistake - by playing with csv... issue can be closed.

[pombredanne]

Thanks for the report!
Scanning d:\PySide6_Essentials-6.3.0-cp36-abi3-win_amd64\PySide6\qml\QtQuick\Controls\Universal\MenuSeparator.qml with the latest/newest beta 31.0.0b3 I have mostly correct but not perfect results, so this is a bug:
I have this:

    -   tool_name: scancode-toolkit
        tool_version: 31.0.0b3
        options:
            input:
                - ./PySide6_Essentials-6.3.0-cp36-abi3-win_amd64.whl-extract/PySide6/qml/QtQuick/Controls/Universal/MenuSeparator.qml
            --license: yes
            --license-text: yes
            --license-text-diagnostics: yes
            --yaml: '-'
        notice: |
            Generated with ScanCode and provided on an "AS IS" BASIS, WITHOUT WARRANTIES
            OR CONDITIONS OF ANY KIND, either express or implied. No content created from
            ScanCode should be considered or used as legal advice. Consult an Attorney
            for any legal advice.
            ScanCode is a free software code scanning tool from nexB Inc. and others.
            Visit https://github.com/nexB/scancode-toolkit/ for support and download.
        start_timestamp: '2022-05-02T063034.506507'
        end_timestamp: '2022-05-02T063036.998398'
        output_format_version: 2.0.0
        duration: '2.4919016361236572'
        message:
        errors: []
        warnings: []
        extra_data:
            system_environment:
                operating_system: linux
                cpu_architecture: 64
                platform: Linux-4.15.0-176-generic-x86_64-with-glibc2.23
                platform_version: '#185~16.04.1-Ubuntu SMP Mon Apr 4 10:41:35 UTC 2022'
                python_version: "3.9.10 (main, Jan 29 2022, 10:01:49) \n[GCC 5.4.0 20160609]"
            spdx_license_list_version: '3.16'
            files_count: 1
files:
    -   path: MenuSeparator.qml
        type: file
        licenses:
            -   key: lgpl-2.0-plus
                score: '100.0'
                name: GNU Library General Public License 2.0 or later
                short_name: LGPL 2.0 or later
                category: Copyleft Limited
                is_exception: no
                is_unknown: no
                owner: Free Software Foundation (FSF)
                homepage_url: http://www.gnu.org/licenses/old-licenses/lgpl-2.0.html
                text_url: http://www.gnu.org/licenses/old-licenses/lgpl-2.0-standalone.html
                reference_url: https://scancode-licensedb.aboutcode.org/lgpl-2.0-plus
                scancode_text_url: https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/licenses/lgpl-2.0-plus.LICENSE
                scancode_data_url: https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/licenses/lgpl-2.0-plus.yml
                spdx_license_key: LGPL-2.0-or-later
                spdx_url: https://spdx.org/licenses/LGPL-2.0-or-later
                start_line: 8
                end_line: 8
                matched_rule:
                    identifier: lgpl-2.0-plus_51.RULE
                    license_expression: lgpl-2.0-plus
                    licenses:
                        - lgpl-2.0-plus
                    referenced_filenames: []
                    is_license_text: no
                    is_license_notice: no
                    is_license_reference: no
                    is_license_tag: yes
                    is_license_intro: no
                    has_unknown: no
                    matcher: 2-aho
                    rule_length: 2
                    matched_length: 2
                    match_coverage: '100.0'
                    rule_relevance: 100
                matched_text: LICENSE:LGPL$
            -   key: commercial-license
                score: '100.0'
                name: Commercial License
                short_name: Commercial License
                category: Commercial
                is_exception: no
                is_unknown: no
                owner: Unspecified
                homepage_url:
                text_url:
                reference_url: https://scancode-licensedb.aboutcode.org/commercial-license
                scancode_text_url: https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/licenses/commercial-license.LICENSE
                scancode_data_url: https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/licenses/commercial-license.yml
                spdx_license_key: LicenseRef-scancode-commercial-license
                spdx_url: https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/licenses/commercial-license.LICENSE
                start_line: 9
                end_line: 15
                matched_rule:
                    identifier: commercial-license_24.RULE
                    license_expression: commercial-license
                    licenses:
                        - commercial-license
                    referenced_filenames: []
                    is_license_text: no
                    is_license_notice: yes
                    is_license_reference: no
                    is_license_tag: no
                    is_license_intro: no
                    has_unknown: no
                    matcher: 2-aho
                    rule_length: 67
                    matched_length: 67
                    match_coverage: '100.0'
                    rule_relevance: 100
                matched_text: "Commercial License Usage\r\n** Licensees holding valid commercial\
                    \ Qt licenses may use this file in\r\n** accordance with the commercial\
                    \ license agreement provided with the\r\n** Software or, alternatively,\
                    \ in accordance with the terms contained in\r\n** a written agreement between\
                    \ you and The Qt Company. For licensing terms\r\n** and conditions see https://www.qt.io/terms-conditions.\
                    \ For further\r\n** information use the contact form at https://www.qt.io/contact-us."
            -   key: digia-qt-commercial
                score: '69.86'
                name: Digia Qt Commercial License Usage
                short_name: Digia Qt Commercial
                category: Commercial
                is_exception: no
                is_unknown: no
                owner: Digia
                homepage_url: http://qt.digia.com/licensing
                text_url:
                reference_url: https://scancode-licensedb.aboutcode.org/digia-qt-commercial
                scancode_text_url: https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/licenses/digia-qt-commercial.LICENSE
                scancode_data_url: https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/licenses/digia-qt-commercial.yml
                spdx_license_key: LicenseRef-scancode-digia-qt-commercial
                spdx_url: https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/licenses/digia-qt-commercial.LICENSE
                start_line: 9
                end_line: 28
                matched_rule:
                    identifier: digia-qt-commercial_or_lgpl-3.0_or_gpl-2.0_1.RULE
                    license_expression: digia-qt-commercial OR lgpl-3.0 OR gpl-2.0
                    licenses:
                        - digia-qt-commercial
                        - lgpl-3.0
                        - gpl-2.0
                    referenced_filenames: []
                    is_license_text: no
                    is_license_notice: yes
                    is_license_reference: no
                    is_license_tag: no
                    is_license_intro: no
                    has_unknown: no
                    matcher: 3-seq
                    rule_length: 209
                    matched_length: 146
                    match_coverage: '69.86'
                    rule_relevance: 100
                matched_text: "Commercial License Usage\r\n** Licensees holding valid commercial\
                    \ Qt licenses may use this file in\r\n** accordance with the commercial\
                    \ license agreement provided with the\r\n** Software or, alternatively,\
                    \ in accordance with the terms contained in\r\n** a written agreement between\
                    \ you and [The] [Qt] [Company]. For licensing terms\r\n** and conditions\
                    \ see [https]://[www].[qt].[io]/[terms]-[conditions]. [For] [further]\r\n\
                    ** [information] [use] [the] [contact] [form] [at] [https]://[www].[qt].[io]/contact-us.\r\
                    \n**\r\n** GNU Lesser General Public License Usage\r\n** Alternatively,\
                    \ this file may be used under the terms of the GNU Lesser\r\n** General\
                    \ Public License version 3 as published by the Free Software\r\n** Foundation\
                    \ and appearing in the file LICENSE.[LGPL3] included in the\r\n** packaging\
                    \ of this file. Please review the following information to\r\n** ensure\
                    \ the GNU Lesser General Public License version 3 requirements\r\n** will\
                    \ be met: https://www.gnu.org/licenses/lgpl-[3].[0].html.\r\n**\r\n** GNU\
                    \ General Public License Usage\r\n** Alternatively, this file may be used\
                    \ under the terms of the GNU\r\n** General Public License version 2.0 or\
                    \ ([at] [your] [option]) the GNU General\r\n** Public license version"
            -   key: lgpl-3.0
                score: '69.86'
                name: GNU Lesser General Public License 3.0
                short_name: LGPL 3.0
                category: Copyleft Limited
                is_exception: no
                is_unknown: no
                owner: Free Software Foundation (FSF)
                homepage_url: http://www.gnu.org/licenses/lgpl-3.0.html
                text_url: http://www.gnu.org/licenses/lgpl-3.0-standalone.html
                reference_url: https://scancode-licensedb.aboutcode.org/lgpl-3.0
                scancode_text_url: https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/licenses/lgpl-3.0.LICENSE
                scancode_data_url: https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/licenses/lgpl-3.0.yml
                spdx_license_key: LGPL-3.0-only
                spdx_url: https://spdx.org/licenses/LGPL-3.0-only
                start_line: 9
                end_line: 28
                matched_rule:
                    identifier: digia-qt-commercial_or_lgpl-3.0_or_gpl-2.0_1.RULE
                    license_expression: digia-qt-commercial OR lgpl-3.0 OR gpl-2.0
                    licenses:
                        - digia-qt-commercial
                        - lgpl-3.0
                        - gpl-2.0
                    referenced_filenames: []
                    is_license_text: no
                    is_license_notice: yes
                    is_license_reference: no
                    is_license_tag: no
                    is_license_intro: no
                    has_unknown: no
                    matcher: 3-seq
                    rule_length: 209
                    matched_length: 146
                    match_coverage: '69.86'
                    rule_relevance: 100
                matched_text: "Commercial License Usage\r\n** Licensees holding valid commercial\
                    \ Qt licenses may use this file in\r\n** accordance with the commercial\
                    \ license agreement provided with the\r\n** Software or, alternatively,\
                    \ in accordance with the terms contained in\r\n** a written agreement between\
                    \ you and [The] [Qt] [Company]. For licensing terms\r\n** and conditions\
                    \ see [https]://[www].[qt].[io]/[terms]-[conditions]. [For] [further]\r\n\
                    ** [information] [use] [the] [contact] [form] [at] [https]://[www].[qt].[io]/contact-us.\r\
                    \n**\r\n** GNU Lesser General Public License Usage\r\n** Alternatively,\
                    \ this file may be used under the terms of the GNU Lesser\r\n** General\
                    \ Public License version 3 as published by the Free Software\r\n** Foundation\
                    \ and appearing in the file LICENSE.[LGPL3] included in the\r\n** packaging\
                    \ of this file. Please review the following information to\r\n** ensure\
                    \ the GNU Lesser General Public License version 3 requirements\r\n** will\
                    \ be met: https://www.gnu.org/licenses/lgpl-[3].[0].html.\r\n**\r\n** GNU\
                    \ General Public License Usage\r\n** Alternatively, this file may be used\
                    \ under the terms of the GNU\r\n** General Public License version 2.0 or\
                    \ ([at] [your] [option]) the GNU General\r\n** Public license version"
            -   key: gpl-2.0
                score: '69.86'
                name: GNU General Public License 2.0
                short_name: GPL 2.0
                category: Copyleft
                is_exception: no
                is_unknown: no
                owner: Free Software Foundation (FSF)
                homepage_url: http://www.gnu.org/licenses/gpl-2.0.html
                text_url: http://www.gnu.org/licenses/gpl-2.0.txt
                reference_url: https://scancode-licensedb.aboutcode.org/gpl-2.0
                scancode_text_url: https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/licenses/gpl-2.0.LICENSE
                scancode_data_url: https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/licenses/gpl-2.0.yml
                spdx_license_key: GPL-2.0-only
                spdx_url: https://spdx.org/licenses/GPL-2.0-only
                start_line: 9
                end_line: 28
                matched_rule:
                    identifier: digia-qt-commercial_or_lgpl-3.0_or_gpl-2.0_1.RULE
                    license_expression: digia-qt-commercial OR lgpl-3.0 OR gpl-2.0
                    licenses:
                        - digia-qt-commercial
                        - lgpl-3.0
                        - gpl-2.0
                    referenced_filenames: []
                    is_license_text: no
                    is_license_notice: yes
                    is_license_reference: no
                    is_license_tag: no
                    is_license_intro: no
                    has_unknown: no
                    matcher: 3-seq
                    rule_length: 209
                    matched_length: 146
                    match_coverage: '69.86'
                    rule_relevance: 100
                matched_text: "Commercial License Usage\r\n** Licensees holding valid commercial\
                    \ Qt licenses may use this file in\r\n** accordance with the commercial\
                    \ license agreement provided with the\r\n** Software or, alternatively,\
                    \ in accordance with the terms contained in\r\n** a written agreement between\
                    \ you and [The] [Qt] [Company]. For licensing terms\r\n** and conditions\
                    \ see [https]://[www].[qt].[io]/[terms]-[conditions]. [For] [further]\r\n\
                    ** [information] [use] [the] [contact] [form] [at] [https]://[www].[qt].[io]/contact-us.\r\
                    \n**\r\n** GNU Lesser General Public License Usage\r\n** Alternatively,\
                    \ this file may be used under the terms of the GNU Lesser\r\n** General\
                    \ Public License version 3 as published by the Free Software\r\n** Foundation\
                    \ and appearing in the file LICENSE.[LGPL3] included in the\r\n** packaging\
                    \ of this file. Please review the following information to\r\n** ensure\
                    \ the GNU Lesser General Public License version 3 requirements\r\n** will\
                    \ be met: https://www.gnu.org/licenses/lgpl-[3].[0].html.\r\n**\r\n** GNU\
                    \ General Public License Usage\r\n** Alternatively, this file may be used\
                    \ under the terms of the GNU\r\n** General Public License version 2.0 or\
                    \ ([at] [your] [option]) the GNU General\r\n** Public license version"
            -   key: kfqf-accepted-gpl
                score: '100.0'
                name: KFQF Accepted GPL
                short_name: KFQF Accepted GPL
                category: Copyleft
                is_exception: yes
                is_unknown: no
                owner: KDE Project
                homepage_url: https://github.com/KDE/licensedigger/blob/master/licensetexts/LicenseRef-KFQF-Accepted-GPL.txt
                text_url:
                reference_url: https://scancode-licensedb.aboutcode.org/kfqf-accepted-gpl
                scancode_text_url: https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/licenses/kfqf-accepted-gpl.LICENSE
                scancode_data_url: https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/licenses/kfqf-accepted-gpl.yml
                spdx_license_key: LicenseRef-scancode-kfqf-accepted-gpl
                spdx_url: https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/licenses/kfqf-accepted-gpl.LICENSE
                start_line: 26
                end_line: 34
                matched_rule:
                    identifier: kfqf-accepted-gpl.LICENSE
                    license_expression: kfqf-accepted-gpl
                    licenses:
                        - kfqf-accepted-gpl
                    referenced_filenames: []
                    is_license_text: yes
                    is_license_notice: no
                    is_license_reference: no
                    is_license_tag: no
                    is_license_intro: no
                    has_unknown: no
                    matcher: 2-aho
                    rule_length: 102
                    matched_length: 102
                    match_coverage: '100.0'
                    rule_relevance: 100
                matched_text: "Alternatively, this file may be used under the terms of the GNU\r\
                    \n** General Public License version 2.0 or (at your option) the GNU General\r\
                    \n** Public license version 3 or any later version approved by the KDE Free\r\
                    \n** Qt Foundation. The licenses are as published by the Free Software\r\
                    \n** Foundation and appearing in the file LICENSE.GPL2 and LICENSE.GPL3\r\
                    \n** included in the packaging of this file. Please review the following\r\
                    \n** information to ensure the GNU General Public License requirements will\r\
                    \n** be met: https://www.gnu.org/licenses/gpl-2.0.html and\r\n** https://www.gnu.org/licenses/gpl-3.0.html."
        license_expressions:
            - lgpl-2.0-plus
            - commercial-license
            - digia-qt-commercial OR lgpl-3.0 OR gpl-2.0
            - kfqf-accepted-gpl
        percentage_of_license_text: '73.21'
        scan_errors: []

I would expect instead only something such as: qt-io-commercial OR lgpl-3.0 OR kfqf-accepted-gpl
FWIW, the many incarnations of QT as a company have always been playing it cute (pun intended) with license notices, coming with new ways to desribed their licensing every now and then.

[pombredanne]

@yahym unrelated, you pasted this nice screenshot... I am curious about which app this comes from!
Do you mind to elaborate?

[yahym]

Just a simple excel pivot table from csv export... my filtering was wrong and I missed some data.

I attached the exports after the scan (see xlsm file). Seems the detection worked properly on scancode side.
PySide6_essentials.ZIP

After filters removal, looks that both GPL/LGPL were detected:

Thank you for your quick answer.

[pombredanne]

@yahym Thanks for the reply. Let me keep this open as there are still some detections that we can improve there.