Parse and validate SPDX license expressions by JonoYang · Pull Request #58 · aboutcode-org/license-expression

JonoYang · 2021-06-04T20:23:19Z

This PR adds a new method to Licensing called validate() to address #56. This method returns an ExpressionInfo object that contains information about a license expression that is passed into Licensing.validate(). Utility functions build_licensing() and build_spdx_licensing() has been added to make it more convenient to use a Licensing object that has already been preloaded with valid license keys so license keys in expressions can be validated. A copy of the license key index from https://scancode-licensedb.aboutcode.org/index.json has been vendored as well.

* Index SPDX license keys instead of scancode license keys * Modify code to do lookups using SPDX license keys Signed-off-by: Jono Yang <jyang@nexb.com>

Signed-off-by: Jono Yang <jyang@nexb.com>

* Create functions that loads a Licensing object with SPDX licenses Signed-off-by: Jono Yang <jyang@nexb.com>

* Refactor validate() to call parse() rather than using the code from parse() Signed-off-by: Jono Yang <jyang@nexb.com>

* Return license validation results in ExpressionInfo object Signed-off-by: Jono Yang <jyang@nexb.com>

Signed-off-by: Jono Yang <jyang@nexb.com>

* Make helper functions for loading license keys Signed-off-by: Jono Yang <jyang@nexb.com>

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>

* Add test for get_license_key_info Signed-off-by: Jono Yang <jyang@nexb.com>

pombredanne

Thanks! See a few nitpickings for your considerations

pombredanne · 2021-06-05T15:30:54Z

+      - If a license symbol in the license expression is a license exception,
+        then that license symbol will be appended here.
+    """
+    def __init__(self):


Why no accepting the attributes as arguments to the consrtuctor?

I had issues when I did the following:

def __init__( self, original_license_expression, normalized_license_expression='', errors=[], valid_symbols=[], valid_exception_symbols=[], invalid_symbols=[]): self.original_license_expression = original_license_expression self.normalized_license_expression = normalized_license_expression self.errors = errors self.valid_symbols = valid_symbols self.valid_exception_symbols = valid_exception_symbols self.invalid_symbols = invalid_symbols

During the license validation tests, new instances of ExpressionInfo would have error messages from the other instances of ExpressionInfo previously made. I wasn't sure what was the best way to mimic a dataclass in Python 3.6.

* Set original license expression in ExpressionInfo * Set vendored licensedb info location as a global * Create function that loads license index json * Update tests Signed-off-by: Jono Yang <jyang@nexb.com>

Signed-off-by: Jono Yang <jyang@nexb.com>

pombredanne

Thanks... see my final few nits for your consideration.

pombredanne · 2021-06-08T08:21:30Z

+    def test_validation_invalid_license_exception(self):
+        result = self.licensing.validate('Apache-2.0 WITH MIT')
+        assert result.original_license_expression == 'Apache-2.0 WITH MIT'
+        assert result.normalized_license_expression == ''


The exception symbol is not correct here, but the normalized expression still parses alright and this is fine IMHO

Suggested change

assert result.normalized_license_expression == ''

assert result.normalized_license_expression == 'Apache-2.0 WITH MIT'

* Add repr to ExpressionInfo class * Remove valid_symbols and valid_exception_symbols from ExpressionInfo * Update vendored licensedb index * Avoid indexing deprecated licenses * Update tests Signed-off-by: Jono Yang <jyang@nexb.com>

pombredanne · 2021-06-09T17:21:25Z

+
+        # Check `expression` keys (validate)
+        try:
+            self.validate_license_keys(expression)


Suggested change

self.validate_license_keys(expression)

self.validate_license_keys(parsed_expression)

pombredanne

I think you can conflate the first two try/except blocks in validate() and just use the second block (passing strict=strict)

* Add new test that uses license exception as normal license key Signed-off-by: Jono Yang <jyang@nexb.com>

* We keep track of invalid license symbols from syntax errors Signed-off-by: Jono Yang <jyang@nexb.com>

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>

pombredanne

LGTM!
Thanks!

Remove macos 10.14 job from azure-pipelines.yml

JonoYang and others added 9 commits June 1, 2021 16:47

Begin to add spdx-mode to Licensing #56

38a9593

* Index SPDX license keys instead of scancode license keys * Modify code to do lookups using SPDX license keys Signed-off-by: Jono Yang <jyang@nexb.com>

Remove spdx argument from functions/methods #56

3650ea9

Signed-off-by: Jono Yang <jyang@nexb.com>

Create validate method on Licensing #56

3601e2e

* Create functions that loads a Licensing object with SPDX licenses Signed-off-by: Jono Yang <jyang@nexb.com>

Vendor license key index #56

97d1fe6

* Refactor validate() to call parse() rather than using the code from parse() Signed-off-by: Jono Yang <jyang@nexb.com>

Update tests and documentation #56

4521921

* Return license validation results in ExpressionInfo object Signed-off-by: Jono Yang <jyang@nexb.com>

Add test for build_spdx_licensing #56

25d29f1

Signed-off-by: Jono Yang <jyang@nexb.com>

Add test for build_spdx_licensing() #56

5a1c830

* Make helper functions for loading license keys Signed-off-by: Jono Yang <jyang@nexb.com>

Merge lastest master

4ab809f

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>

Move ExpressionInfo class definition #56

8d198cd

* Add test for get_license_key_info Signed-off-by: Jono Yang <jyang@nexb.com>

JonoYang requested a review from pombredanne June 4, 2021 20:24

pombredanne requested changes Jun 5, 2021

View reviewed changes

JonoYang added 2 commits June 7, 2021 19:14

Address PR comments #56

ec9242f

* Set original license expression in ExpressionInfo * Set vendored licensedb info location as a global * Create function that loads license index json * Update tests Signed-off-by: Jono Yang <jyang@nexb.com>

Create validate_license_key method #56

f6c8fbd

Signed-off-by: Jono Yang <jyang@nexb.com>

JonoYang force-pushed the 56-parse-and-validate-spdx-v2 branch from ee70004 to f6c8fbd Compare June 8, 2021 02:56

pombredanne requested changes Jun 8, 2021

View reviewed changes

JonoYang force-pushed the 56-parse-and-validate-spdx-v2 branch 4 times, most recently from 4128774 to 6f71243 Compare June 8, 2021 21:33

Update ExpressionInfo #56

ac80a21

* Add repr to ExpressionInfo class * Remove valid_symbols and valid_exception_symbols from ExpressionInfo * Update vendored licensedb index * Avoid indexing deprecated licenses * Update tests Signed-off-by: Jono Yang <jyang@nexb.com>

JonoYang force-pushed the 56-parse-and-validate-spdx-v2 branch from 6f71243 to ac80a21 Compare June 8, 2021 23:54

pombredanne reviewed Jun 9, 2021

View reviewed changes

pombredanne approved these changes Jun 9, 2021

View reviewed changes

JonoYang added 2 commits June 9, 2021 11:58

Parse expression once in validate #56

8d86c4b

* Add new test that uses license exception as normal license key Signed-off-by: Jono Yang <jyang@nexb.com>

Rename invalid_keys to invalid_symbols #56

ef9b2df

* We keep track of invalid license symbols from syntax errors Signed-off-by: Jono Yang <jyang@nexb.com>

pombredanne reviewed Jun 10, 2021

View reviewed changes

Comment thread src/license_expression/__init__.py Outdated

Make aliases a list if empty #56

3487a13

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>

pombredanne approved these changes Jun 10, 2021

View reviewed changes

pombredanne merged commit 2b0c7be into master Jun 10, 2021

pombredanne deleted the 56-parse-and-validate-spdx-v2 branch June 10, 2021 06:30

pombredanne pushed a commit that referenced this pull request May 10, 2022

Merge pull request #58 from nexB/remove-mac-1014

7eb4ca8

Remove macos 10.14 job from azure-pipelines.yml

stefan6419846 mentioned this pull request Jul 4, 2024

Deprecated SPDX licenses are marked as unknown #83

Open

AyanSinhaMahapatra mentioned this pull request Jun 10, 2025

Provide built-in support for SPDX and scancode license expression validation #56

Closed

	assert result.normalized_license_expression == ''
	assert result.normalized_license_expression == 'Apache-2.0 WITH MIT'

	self.validate_license_keys(expression)
	self.validate_license_keys(parsed_expression)

Uh oh!

Uh oh!

Conversation

JonoYang commented Jun 4, 2021

Uh oh!

pombredanne left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pombredanne Jun 5, 2021

Choose a reason for hiding this comment

Uh oh!

JonoYang Jun 8, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pombredanne left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pombredanne Jun 8, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pombredanne Jun 9, 2021

Choose a reason for hiding this comment

Uh oh!

pombredanne left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pombredanne left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants