Repository navigation
Conversation
Replaces BufferedWriter's bytearray + _write_lock with a _Nibbler buffer manager built around _Morsel work units. Adds FileIO._writev so the nibbler can issue a single writev(2) for multi-buffer flushes when available, falling back to coalesce+write otherwise. The Nibbler holds its lock end-to-end across raw I/O, matching the pre-existing cross-thread write ordering guarantee. A future two-lock split (short buffer lock + I/O lock) is sketched in the class docstring as the intended path for releasing the I/O lock once profiling justifies it. Also restores f.close() in BufferSizeTest.try_one's nested finally and skips test_slow_close_from_thread pending a way to validate pythongh-76157 / pythongh-31976 that does not assume the old lock-across-I/O behavior. See: https://discuss.python.org/t/reworking-buffered-i-o-in-cpython/90019 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Grow _Nibbler to be the actual buffered I/O implementation that
BufferedReader / BufferedWriter / BufferedRandom subclass directly,
mirroring how the _io C struct `buffered` backs all three Python types.
The class absorbs the responsibilities of the deleted _BufferedIOMixin
plus the previous writer-only _Nibbler, and gains a full read API.
- One concrete implementation. read/write/seek/tell/close/etc. all
live on _Nibbler. Subclasses define __init__ validation and override
readable()/writable() to gate which methods are permitted (the same
pattern _io uses in C).
- Mode flag (READING/WRITING) tracked under the lock. The first write
flips READING -> WRITING; the first read after a write flushes and
flips back. Mode switches happen atomically inside the nibbler so
BufferedRandom no longer needs ad-hoc flush/seek-back coordination
in every override.
- buffer_size=0 supported on both reader and writer. Writer flushes
on every append. Reader passes through to raw for read/read1/
readinto. peek raises UnsupportedOperation since there is no
buffer to peek into.
- BufferedRandom keeps the (BufferedWriter, BufferedReader) MRO; the
cooperative super() chain produces a single _Nibbler.__init__ call.
Random must override readable()/writable() to True since MRO would
otherwise inherit False from BufferedWriter.
- BufferedRWPair is unchanged - it composes two independent streams
so the unified-buffer model does not apply.
Test changes: PyBufferedReader.test_constructor's buffer_size=0
ValueError check is gated to the C implementation (matches the
pre-existing gating for the writer side).
The follow-on stage routes open(buffering=0) through BufferedIO so
all open() forms retry partial writes uniformly.
See: https://discuss.python.org/t/reworking-buffered-i-o-in-cpython/90019
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Stage 2 of the unified-Nibbler work. With BufferedReader / BufferedWriter
/ BufferedRandom now accepting buffer_size=0 (Stage 1), open(buffering=0)
no longer needs to return a raw FileIO - it can return a BufferedIO with
buffer_size=0 that retries partial writes. This unifies the open() forms
so all of them either consume all data or raise (PEP-475 EINTR plus the
write-retry loop), removing FileIO's POSIX-style "check the return size"
quirk for callers that just used buffering=0 to avoid buffering.
- open(buffering=0, mode='rb'/'wb'/'r+b'/'ab'/'xb') now returns a
BufferedReader / BufferedWriter / BufferedRandom with buffer_size=0
(was: raw FileIO for binary, ValueError for text).
- The text-mode rejection of buffering=0 stays.
- Add _Nibbler.readall() so BufferedWriter (now what's returned for
open('wb', 0)) raises UnsupportedOperation through self.read(-1)
when callers reach for the FileIO-style readall, matching the prior
error type.
- Update test_abc_inheritance / test_abc_inheritance_official to
reflect that _pyio.open(buffering=0, 'wb') returns a BufferedIOBase
instance, not a RawIOBase. The C io.open is unchanged so the assertion
is gated on which implementation the test is exercising.
This is _pyio only. The C _io.open will need the same routing change
when the C nibbler lands.
See: https://discuss.python.org/t/reworking-buffered-i-o-in-cpython/90019
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.