Newer tlds not supported

[armijnhemel]

Description

The current e-mail regular expression cannot catch the latest (and greatest?) TLDs. There are many these days:

https://en.wikipedia.org/wiki/List_of_Internet_top-level_domains

but scancode limits it to something either 2, 3 or 4 characters:

def emails_regex():
    return re.compile('\\b[A-Z0-9._%-]+@[A-Z0-9.-]+\\.[A-Z]{2,4}\\b', re.IGNORECASE)

[armijnhemel]

Of course, a follow up question would be: is this information actually useful to extract?

[armijnhemel]

Also, this module might be worth adopting: https://pypi.org/project/email-validator/

[macayu17]

I can take a small pass at this. My read is that find_emails() only needs to extract email-like clues, so I would start by updating the existing regex to support longer TLDs and add a focused regression case.

Also, would you prefer that smaller regex-only fix, or do you want this to evaluate adopting email-validator instead?

[armijnhemel]

email-validator is there and seems to be tested thoroughly, so probably what I would do is capture a little bit too much and then let email-validator do the verification. @pombredanne @AyanSinhaMahapatra what do you think?

[AyanSinhaMahapatra]

@armijnhemel thanks for the issue. I think this is a good idea potentially, we'll have to check a bit more details to make sure everything works okay.

email-validator has been relicensed since I last checked them out: cc0 -> unlicense. 👀

We should:

• craft some tests out of https://en.wikipedia.org/wiki/List_of_Internet_top-level_domains
• evaluate https://github.com/JoshData/python-email-validator/ vs https://github.com/michaelherold/pyIsEmail
• https://pypi.org/project/dnspython/ is a new dependency addition for both cases, looks okay to me, no other deps