Repository navigation
geotiff tests: consolidate codec round-trip cluster (jpeg/jpeg2000/lerc/lz4) - #2450
Merged
Merged
Conversation
Fold the codec round-trip files into the new unit/test_codec_roundtrip.py. unit/test_compression.py keeps its PackBits-kernel-only scope as its docstring describes. Files folded: - test_jpeg.py - test_jpeg2000.py - test_lerc.py - test_lerc_max_z_error.py - test_lerc_valid_mask.py - test_lz4.py - test_lz4_compression_level_2026_05_11.py - test_compression_docstring_1644.py - test_compression_level.py Net file delta: 9 deleted, 1 added => -8 files. Test count unchanged (112 in scope); descriptive parametrise ids (e.g. jpeg2000[uint16-tiled], lerc[float32-stripped], lz4-level[16]) with no issue numbers in test or filenames. LERC max_z_error numerical tolerances preserved verbatim. JPEG2000 and LERC sections use pytest.importorskip / skip markers for the optional codec dependencies. CLUSTER_AUDIT_CODEC_ROUNDTRIP.md maps old file::test to new file::test_id; it lands on this branch and is deleted in a pre-merge commit so it does not reach main. Part of long-tail epic #2424 / cluster #2425 PR A.
brendancol
commented
May 26, 2026
brendancol
left a comment
Contributor
Author
There was a problem hiding this comment.
PR Review: geotiff tests: consolidate codec round-trip cluster
Tests-only change. No edits to xrspatial/geotiff/ source modules. 9 top-level files folded into a new unit/test_codec_roundtrip.py. 112 tests preserved (same baseline count).
Blockers
None.
Suggestions
xrspatial/geotiff/tests/unit/test_codec_roundtrip.py:1308-1315: the comment for_GPU_FALLBACK_CODECSnow says "the JPEG round-trip is covered with appropriate uint8 RGB data elsewhere" without naming where. The original file pointed attest_gpu_writer_compression_modes_2026_05_11.py. Since GPU codec consolidation is owned by #2438, the most accurate phrasing here is "covered by the GPU codec cluster (issue #2438)". One-line wording tweak.xrspatial/geotiff/tests/unit/test_codec_roundtrip.py:96-100and:475-481: there are two helpers named_make_compressibleand_make_compressible_largewith slightly different shapes ((128,128) vs (512,512)) and noise levels. Keeping them separate is correct (the lz4 size-effect test uses a smaller surface than the zstd/deflate one for runtime reasons), but the naming is easy to misread. Renaming the module-level helper to_make_compressible_smooth_smallor inlining it intoTestLz4CompressionLevelwould remove the ambiguity.xrspatial/geotiff/tests/unit/test_codec_roundtrip.py:617-626: the LERC mask predicate closure capturesinvalid_positionsvia the default-arg pattern (def invalid_pred(a, positions=invalid_positions)). Good defensive choice. Worth a one-line comment explaining the bound-default pattern is intentional; without it a future refactor could collapse it to a free closure and reintroduce the late-binding bug.
Nits
xrspatial/geotiff/tests/unit/test_codec_roundtrip.py:53:from xrspatial.geotiff._compression import (...)is a long parenthesised import. The leading itemsCOMPRESSION_JPEG, COMPRESSION_JPEG2000, COMPRESSION_LERC, COMPRESSION_LZ4are constants and the rest are functions; isort already keeps them in one block but a blank-line break after the constants would make the import easier to read.xrspatial/geotiff/tests/unit/test_codec_roundtrip.py:200-204:TestPublicToGeotiffJpegRejectedis a class with a single test method. Both the original and the consolidated version use a class; nothing wrong with it, just notable.xrspatial/geotiff/tests/unit/test_codec_roundtrip.py:1098, :1110:path = str(tmp_path / "lz4_bad.tif")is shared across the parametrised cases. pytest'stmp_pathis unique per test invocation, so there is no real collision risk, but the project memory note recommends including unique tokens in tmp filenames. Either include the level in the filename (e.g.f"lz4_bad_{level}.tif") or leave a comment saying tmp_path already provides isolation.CLUSTER_AUDIT_CODEC_ROUNDTRIP.md: scheduled for deletion on the pre-merge commit. The file header already says so; flagging here only so the deletion does not get forgotten.
What looks good
- LERC
max_z_errornumerical tolerances preserved verbatim (budget + 1e-7slack, bit-exact lossless viaassert_array_equal). pytest.importorskip/ module-level skip markers kept for the optional codec libraries (glymur, lerc, lz4, rasterio).- Parametrise ids are descriptive and free of issue numbers (
lerc[float32-stripped],lz4-level[16],lerc-mask[uint16-sentinel]). - The
to_geotiff(compression='jpeg')rejection contract is still pinned (TestPublicToGeotiffJpegRejected). - The rasterio-only GDAL tiled JPEG class is still skip-gated at the class level, so the non-rasterio environment still exercises the codec-only tests above it.
- The
lerc_writer_with_maskmonkeypatch fixture is preserved with the same predicate-callable indirection. - Audit file maps every old
file::testto the newfile::test_id.
Checklist
- Tests-only; no source changes
- All implemented backends produce consistent results
- NaN handling is correct (LERC NaN-nodata case still pinned)
- Edge cases are covered by tests (boundary levels, empty payloads, single pixel)
- Optional-dependency skip markers preserved
- No premature materialization or unnecessary copies
- No benchmark needed (test consolidation only)
- No README feature matrix change needed
- Docstrings present on every consolidated class
…er tmp paths (#2446) - _GPU_FALLBACK_CODECS comment now names cluster issue #2438 for the JPEG GPU coverage referral, replacing the dangling pointer to test_gpu_writer_compression_modes_2026_05_11.py. - Module-level _make_compressible renamed to _make_compressible_smooth_small; docstring explains the size split vs TestCompressionLevelEffect._make_compressible_large. - TestLercTiffRoundTripWithMask predicate closure: comment explaining why ``positions=invalid_positions`` is intentional (late-binding guard). - Out-of-range path filenames now include the codec/level token so a future suite-wide xdist run cannot collide on tmp_path names. - _compression imports grouped with section comments (constants / availability flags / functions) so the long import is easier to scan. All 112 tests still pass; no behaviour change.
brendancol
commented
May 26, 2026
brendancol
left a comment
Contributor
Author
There was a problem hiding this comment.
Follow-up review after nit pass
Disposition of original findings:
Suggestions
- GPU codec referral in
_GPU_FALLBACK_CODECScomment: fixed in 26657e5. Now names the GPU cluster issue #2438 instead of pointing at a file that the GPU cluster will rename. _make_compressible/_make_compressible_largeambiguity: fixed in 26657e5. Module-level helper renamed to_make_compressible_smooth_small; docstring explains the size split.- Default-arg late-binding guard in
invalid_pred: fixed in 26657e5. Comment added explaining the bind is intentional.
Nits
_compressionimport grouping: fixed in 26657e5. Section comments separate constants / availability flags / functions.TestPublicToGeotiffJpegRejectedsingle-method class: dismissed. The original module used a class wrapper here too; flipping to a free function is stylistic preference, not a fix.- Unique tmp_path filenames for parametrise: fixed in 26657e5.
lz4_bad.tif,lz4_dask_bad.tif, and the genericbad.tifnow include codec/level tokens. CLUSTER_AUDIT_CODEC_ROUNDTRIP.mddeletion reminder: tracked. Will be removed in the pre-merge commit on this branch.
Other notes
- Merged
origin/main(commit fe16afe) without conflicts. The merge only added an unrelated nodata-rejection test file at the top level. - 112 tests still pass post-merge and post-fixes.
No new findings. Ready for the pre-merge audit-file deletion commit.
brendancol
added a commit
that referenced
this pull request
May 26, 2026
brendancol
added a commit
that referenced
this pull request
May 26, 2026
* geotiff tests: consolidate predictor cluster (predictor 2 / 3)
Fold six top-level predictor test files into a new
`xrspatial/geotiff/tests/unit/test_predictor.py`, parametrised over
predictor in {2, 3} x dtype x endianness x layout (and multi-sample
for the chunky-band paths).
Files folded:
- test_predictor2_big_endian.py
- test_predictor2_int8.py
- test_predictor3_big_endian.py
- test_predictor3_int_dtype_1933.py
- test_predictor_fp_write_1313.py
- test_predictor_multisample.py
Net file delta: 6 deleted, 1 added => -5 files.
Behaviour preserved:
- predictor=2 round-trips across uint8/int8/uint16/int16/uint32/int32,
big- and little-endian, stripped and tiled.
- predictor=3 round-trips for float32/float64, big- and little-endian,
stripped and tiled (read + writer end-to-end).
- libtiff/GDAL multi-byte sample-level differencing interop.
- Multi-sample predictor=2 GPU parity and predictor=3 multi-sample
TN3 decode (issues #1220, #1247, #1479).
- Predictor=3 + integer SampleFormat validator (#1933).
- normalize_predictor bool|int -> int mapping and rejection branches.
- The opt-in perf gate (predictor=3 within 2x of predictor=2).
Boundary: GPU predictor files
(`test_predictor2_big_endian_gpu_1517.py`,
`test_predictor3_int_dtype_gpu_1933.py`) stay in the GPU cluster
issue #2438. GPU regressions co-located with CPU tests in the old
files move with this consolidation so behaviour-grouped coverage
stays together (matches PR #2451).
Audit: `xrspatial/geotiff/tests/CLUSTER_AUDIT_PREDICTOR.md` maps every
old `file::test` (with parametrise expansion) to the new
`file::test_id`. Deleted on the pre-merge commit before this lands on
main.
Closes #2427.
* Address review nits: parametrise IDs + helper comment (#2427)
- Use `le` / `be` labels instead of `<` / `>` for endianness IDs.
- Flatten nested-bracket IDs (`pred2-libtiff[uint16]` ->
`pred2-libtiff-uint16`, same for the other gpu / writer matrices)
so pytest collection output reads as a single bracket per id.
- Move `imagecodecs_required` skip marker to the top of the file
next to `_gpu_only` so all module-level gates live in one place.
- Annotate `_da_xy` about the trailing-axis indexing on 3D inputs
so a future reader does not "fix" it back to the buggy
pre-consolidation form.
- Reconcile the audit map and the +4 prose with the renamed IDs.
* Delete CLUSTER_AUDIT_PREDICTOR.md pre-merge (#2427)
The audit map served its purpose during review. Per the cluster
convention (see PR #2451 / #2450), the audit file is removed before
merge so it does not land on main.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #2446. PR A of cluster #2425; part of the long-tail epic #2424.
Summary
Folds 9 top-level codec files into a new
unit/test_codec_roundtrip.py.The sibling
unit/test_compression.pykeeps its PackBits-kernel-onlyscope (its docstring already says round-trips live elsewhere), so the
new file is a sibling rather than an extension.
Files folded:
test_jpeg.pytest_jpeg2000.pytest_lerc.pytest_lerc_max_z_error.pytest_lerc_valid_mask.pytest_lz4.pytest_lz4_compression_level_2026_05_11.pytest_compression_docstring_1644.pytest_compression_level.pyNet file delta: 9 deleted, 1 added => -8 files.
What is preserved
max_z_errornumerical tolerances pinned verbatim(
budget + 1e-7slack, bit-exact lossless viaassert_array_equal).pytest.importorskip/skip markers.
to_geotiff(compression='jpeg')rejection contract.write_geotiff_gpudocstring drift guard from geotiff: write_geotiff_gpu compression docstring lists 4 codecs but accepts the full 9 #1644.What changed shape
ids (e.g.
jpeg2000[uint16-tiled],lerc[float32-stripped],lz4-level[16],lerc-mask[uint16-sentinel]).max_z_errorclasses flattened into a single class.dtype × nodata × invalid_positions.Out of scope (PR B, parallel)
PR B owns the photometric cluster and consolidates these into a
different
unit/test_photometric.py:test_miniswhite_nodata_1809.pytest_miniswhite_writer_roundtrip_1836.pytest_signed_miniswhite_rejected_2278.pytest_to_geotiff_allow_internal_only_jpeg_parity.pyGPU codec files (
test_jpeg_gpu_*,test_lerc_valid_mask_gpu.py,test_nvcomp_*,test_nvjpeg*) are scoped to the GPU cluster issue#2438.
Audit trail
xrspatial/geotiff/tests/CLUSTER_AUDIT_CODEC_ROUNDTRIP.mdmaps everyold
file::testto its newfile::test_id. It lives on this branchduring review and is deleted in a final pre-merge commit so it does
not land on
main.Test plan
pytest xrspatial/geotiff/tests/unit/test_codec_roundtrip.py -v(112 passed)
pytest xrspatial/geotiff/tests/unit/ -q(247 passed)pytest xrspatial/geotiff/tests/ -x -q(5778 passed, 68 skipped,6 xfailed)
find xrspatial/geotiff/tests -name 'test_*.py' | wc -ldrops by8 (259 -> 251)