urllib.parse docstrings incomplete

[zware]

BPO	37970
Nosy	@orsenthil, @taleinat, @zware, @nanjekyejoannah, @tmblweed, @idomic
PRs	bpo-37970: update and improve urlparse and urlsplit doc-strings #16458 gh-82151: Improve docs for urllib.parse #18631

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2019-08-28.16:26:39.687>
labels = ['easy', '3.8', '3.9', '3.7', 'library', 'docs']
title = 'urllib.parse docstrings incomplete'
updated_at = <Date 2020-02-23.23:22:55.138>
user = 'https://github.com/zware'

bugs.python.org fields:

activity = <Date 2020-02-23.23:22:55.138>
actor = 'Ido Michael'
assignee = 'docs@python'
closed = False
closed_date = None
closer = None
components = ['Documentation', 'Library (Lib)']
creation = <Date 2019-08-28.16:26:39.687>
creator = 'zach.ware'
dependencies = []
files = []
hgrepos = []
issue_num = 37970
keywords = ['patch', 'newcomer friendly']
message_count = 16.0
messages = ['350668', '350798', '351116', '351135', '351137', '351145', '351338', '353446', '361776', '361777', '361779', '362094', '362096', '362102', '362536', '362550']
nosy_count = 7.0
nosy_names = ['orsenthil', 'taleinat', 'docs@python', 'zach.ware', 'nanjekyejoannah', 'syadlapalli', 'Ido Michael']
pr_nums = ['16458', '18631']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue37970'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9']

Linked PRs

• gh-82151: Update and improve urlsplit docstring #152985
• [3.15] gh-82151: Update and improve urlsplit docstring (GH-152985) #153154
• [3.14] gh-82151: Update and improve urlsplit docstring (#152985) #153155
• [3.13] gh-82151: Update and improve urlsplit docstring (#152985) #153156

[zware]

For example, urlsplit:

>>> from urllib.parse import urlsplit
>>> help(urlsplit)
Help on function urlsplit in module urllib.parse:

urlsplit(url, scheme='', allow_fragments=True)
    Parse a URL into 5 components:
    <scheme>://<netloc>/<path>?<query>#<fragment>
    Return a 5-tuple: (scheme, netloc, path, query, fragment).
    Note that we don't break the components up in smaller bits
    (e.g. netloc is a single string) and we don't expand % escapes.

The current docstring does not describe the scheme or allow_fragments arguments. Also, the note about not splitting netloc is misleading; the components of netloc (username, password, hostname, and port) are available as extra attributes of the returned SplitResult.

urlparse has similar issues; other functions could stand to be checked.

[nanjekyejoannah]

Also, the note about not splitting netloc is misleading; the components > of netloc (username, password, hostname, and port) are available as >extra attributes of the returned SplitResult.

Also, the docs in urllib.parse.rst should also be updated to correct this misleading statement.

[tmblweed]

hello!

I can see that we might want to add documentation for splitting netloc, but I don't understand why we'd have scheme and netloc, but nothing for path and query. What are you suggesting we add for scheme/allow_fragements?

Thanks!

[zware]

I don't understand why we'd have scheme and netloc, but nothing for path and query.

I'm not sure what you mean here; can you please clarify?

What are you suggesting we add for scheme/allow_fragements?

Just a brief description of what effect the arguments actually have on the returned result.

[tmblweed]

I guess what I'm wondering is this:

urlsplit(url, scheme='', allow_fragments=True)
    Parse a URL into 5 components:
    <scheme>://<netloc>/<path>?<query>#<fragment>
    Return a 5-tuple: (scheme, netloc, path, query, fragment).
    Note that we don't break the components up in smaller bits
    (e.g. netloc is a single string) and we don't expand % escapes.
(END)

We don't have details regarding anything, i.e scheme, netloc, path or query or fragments. So I was curious about why we would have more documentation around netloc and scheme and nothing about path and query

Should we be adding information for all(scheme, netloc, path, query, fragment) of them, including extra attributes of the returned SplitResult?

p.s - newbie trying to contribute here

[zware]

I see. I don't think we need to describe each returned component in the docstring; they're some combination of self-explanatory, described in the reference docs, or just industry-standard terms that are easily defined from other sources. I'm not suggesting to add anything to the docstring about the scheme return component, but rather the scheme argument (which is the default value for the scheme return component when it's not found in the url). The subcomponents of netloc should be mentioned because the docstring currently gives the impression that the user has to parse them out for themselves, which is not true.

Off the top of my head, I'd suggest changing the urlsplit docstring to something like:

Parse *url* and return a SplitResult.

SplitResult is a named 5-tuple of the following components:
<scheme>://<netloc>/<path>?<query>#<fragment>

The ``username``, ``password``, ``hostname``, and ``port``
sub-components of ``netloc`` can also be accessed as
attributes of the SplitResult object.

The *scheme* argument provides the default value of the
``scheme`` component when no scheme is found in *url*.

If *allow_fragments* is False, no attempt is made to
separate the ``fragment`` component from the previous
component, which can be either ``path`` or ``query``.

Note that % escapes are not expanded.

[tmblweed]

got it - thanks for the detailed explanation! I'll go ahead and create a PR soon

[idomic]

Committed a PR: #60662

I've read all of the thread and changed the docstring to the latest suggestion by @zach.ware

Ido