#4 Improved expression parsing with names - #6
Merged
Conversation
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
* remove unsued __len__ function * fix issue for trailing unknown strings Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
* now fully supporting WITH expressions (including when mixed with weird OR or when a "with" word is part of a license alias). * support arbitrary possibly ambiguous expressions using a new Aho-Corasick-based lexical scanner populated with known symbols that resolves known license names, keys or aliases at tokenizing time * add new render() function to render symbols. * add new tests, moved several doc tests to proper tests. * drop LicenseRef in favor of plain symbols * improve error and validation handling Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
* a list of plain strings can now be used as a symbols list. * also relaxed unicode requirements for Python 2 as long as a string can convert to unicode without error * refine the render support * implement correct handling of "license_symbols" function and keys with optional unique arg and optional decompose arg to decompose "A with B" in A and B optionally. Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Member
Author
|
@sschuberth you had a good hunch about the travis issue: the python 3 tests were running only on Appveyor. So I hope I fixed the setup for Travis. |
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Member
Author
|
Hum.... The Python 3 tests still do not run, only 2.7 ... on Travis :| I know they pass otherwise so I will likely ignore |
Member
Author
|
I am merging and I entered #8 to fix Travis later. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a a brand new parser using a pure-python Aho-Corasick-based lexical scanner that brings a great flexibility to validate arbitrary expressions including weird things like
GPL-20 or later with classpath exception and mit (the one with sublicense)where each ofGPL-20 or later,classpath exceptionandmit (the one with sublicense)are resolved to proper license symbols (and eventually identifiers).This is made possible by the great library at https://github.com/WojciechMula/pyahocorasick by @WojciechMula . It uses only the simple Python implementation as expressions are small and there is no need for speed here. It has been modified significantly to return also non-matching substrings and demux overlapping results.