Repository navigation
Conversation
Fletcher32.decode copied the payload into a caller-supplied 'out' buffer without checking its size: a smaller buffer led to an out-of-bounds write, while a larger buffer left its tail untouched (stale data). Validate the destination size before the copy and raise ValueError with the expected payload size instead. The zero-length payload case now skips the memcpy entirely, so an empty buffer is never indexed. Add tests for oversized, undersized, exact-size and bytearray destinations.
Author
|
A note on the red CI matrix so it is not mistaken for a regression from this change. The This comes from zfpy Where the matrix is not affected by that dependency issue, the change is fully exercised by the real meson build and the full test suite:
The "Install numcodecs" step (which runs Cython + meson for |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Fletcher32.decodecopies the payload into a caller-providedoutbufferwithout validating the destination size:
Two failure modes:
outsmaller than the decoded payload: out-of-bounds write past the end ofthe destination (heap corruption / crash).
outlarger than the decoded payload: the tail ofoutis left untouched,so stale/uninitialized data is silently returned to the caller.
Fix
Validate the destination size before copying:
out_mvis a 1-Duint8view, so its element count equals the number of bytes.The
if payload_len:guard also keeps a zero-length payload from indexing&out_mv[0]on an empty buffer.Tests
Added to
tests/test_fletcher32.py:test_out_too_large: oversizedoutraisesValueErrorand is not written to(checked by filling it with a sentinel first).
test_out_too_small: undersizedoutraisesValueErrorinstead of overflowing.test_out_correct_size: exact-sizeoutis filled and returned.test_out_bytearray: abytearraydestination is filled correctly.Verification
fletcher32.pyxcleanly.extension, placed it in a temporary copy of the installed package, and ran the
updated
tests/test_fletcher32.py: 10 passed.mainbuild: 2 failed, 8 passed.test_out_too_largeandtest_out_too_smallboth fail withDID NOT RAISE ValueError, confirming that the new tests catch the bug.ruff check tests/test_fletcher32.pyandruff format --check tests/test_fletcher32.pypass.ruffcannot parse.pyxfiles, so the Cython source is verified by compiling it rather than bylinting it.
MSVC/meson toolchain, and the available Python 3.10 environment (with the
matching installed package) predates the project's
requires-python >= 3.12.The standalone module build above exercises the changed code path because
fletcher32.pyxhas no NumPy C-API dependency and the_utilshelpers arecdef inline.Relation to other PRs
payloads; it still copies without a size check. This change is complementary.
Both touch the same few lines, so merging one after the other may need a trivial
conflict resolution; the
if payload_len:guard here overlaps with Fix Fletcher32 for empty buffers #889'sb_len > FOOTER_LENGTHcondition but does not conflict semantically.JenkinsLookup3/Checksum32.Fletcher32is a separate Cythonimplementation and is not covered there.
Checklist
docs/release.md