Improve support for arbitrary license names

[pombredanne]

The current implementation has an arbitrary limitation that an expression with and, or or with keywords that are part of a license name and not part a keyword of the expression would not parse nor resolve correctly.

Now this limitation does not need to exist as when a list of LicenseRef is provided that is otherwise not ambiguous (e.g. with no name resolving to more than one license key) the parsing is not ambiguous.

So I propose to update this to handle properly expressions with such licenses. This would work only when resolution on parsing is requested or when resolution is requested AND that license references are provided.

[pombredanne]

The approach I am taking is this:

• rework the tokenizer to detect tokens (including the with) using an aho-corasick automaton. This will recognize any symbol (including names) keeping the longest of the matches (therefore handling many ambiguities correctly)
• rework the LicenseSymbol class to properly handle construction of pretokenized licenses from above

Beside this, I will add a function to render an expression based on string format templates defined at two levels: a license key and the full expression. This can be used to render links if needed but is needed here to simplify the rendering

[pombredanne]

This is fixed in master. Any unicode "word" character can be accepted with very few restrictions. A license key can contain only words characters, spaces, +, - and _ . punctuations.
This is enforced but could be relaxed in the future. The underlying automaton-based tokenizer can handle pretty much anything you throw at it.