Skip to content

geotiff tests: consolidate predictor cluster (predictor 2 / 3) - #2454

Merged
brendancol merged 4 commits into
mainfrom
worktree-agent-a5f803a0349b781fd
May 26, 2026
Merged

brendancol merged 4 commits into
mainfrom
worktree-agent-a5f803a0349b781fd

Conversation

@brendancol

Copy link
Copy Markdown
Contributor

Closes #2427. Cluster 3 of long-tail epic #2424.

Summary

Fold six top-level predictor test files into a new xrspatial/geotiff/tests/unit/test_predictor.py, parametrised over predictor in {2, 3} x dtype x endianness x layout (and multi-sample for chunky-band paths).

Files folded:

  • test_predictor2_big_endian.py
  • test_predictor2_int8.py
  • test_predictor3_big_endian.py
  • test_predictor3_int_dtype_1933.py
  • test_predictor_fp_write_1313.py
  • test_predictor_multisample.py

Net file delta: 6 deleted, 1 added => -5 files.

What is preserved

What changed shape

  • Parametrise dtype x endianness as a single matrix with descriptive ids (pred2-libtiff[uint16], gpu-pred2[uint16], s3-float64, tiled-deflate-float32, ...).
  • imagecodecs requirement on the predictor=3 read section gated via a section-level marker so the file imports cleanly without it.
  • Issue-number suffixes dropped from test names (covered in the audit map).

Boundary

GPU predictor files
(test_predictor2_big_endian_gpu_1517.py,
test_predictor3_int_dtype_gpu_1933.py) stay in the GPU cluster issue #2438. GPU regressions that lived inside CPU-named files (int8 tiled+stripped, BE GPU, multi-sample GPU parity) move with this consolidation so behaviour-grouped coverage stays together (matches PR #2451).

Audit

xrspatial/geotiff/tests/CLUSTER_AUDIT_PREDICTOR.md maps every old file::test (with parametrise expansion) to its new file::test_id. Deleted in a final pre-merge commit before the PR lands on main.

Verification

  • pytest xrspatial/geotiff/tests/unit/test_predictor.py -v: 80 passed, 1 skipped (perf gate).
  • pytest xrspatial/geotiff/tests/unit/ -q: 364 passed, 1 skipped.
  • pytest xrspatial/geotiff/tests/ -x -q: 5821 passed, 68 skipped, 2 xfailed.
  • find xrspatial/geotiff/tests -name 'test_*.py' | wc -l: 245 (down from 250; -6 deleted +1 added = -5).

Test plan

  • New unit/test_predictor.py passes locally.
  • Full xrspatial/geotiff/tests/ suite passes locally.
  • CI green across numpy / cupy / dask+numpy / dask+cupy.
  • Audit file deleted on the pre-merge commit.

Fold six top-level predictor test files into a new
`xrspatial/geotiff/tests/unit/test_predictor.py`, parametrised over
predictor in {2, 3} x dtype x endianness x layout (and multi-sample
for the chunky-band paths).

Files folded:
- test_predictor2_big_endian.py
- test_predictor2_int8.py
- test_predictor3_big_endian.py
- test_predictor3_int_dtype_1933.py
- test_predictor_fp_write_1313.py
- test_predictor_multisample.py

Net file delta: 6 deleted, 1 added => -5 files.

Behaviour preserved:
- predictor=2 round-trips across uint8/int8/uint16/int16/uint32/int32,
  big- and little-endian, stripped and tiled.
- predictor=3 round-trips for float32/float64, big- and little-endian,
  stripped and tiled (read + writer end-to-end).
- libtiff/GDAL multi-byte sample-level differencing interop.
- Multi-sample predictor=2 GPU parity and predictor=3 multi-sample
  TN3 decode (issues #1220, #1247, #1479).
- Predictor=3 + integer SampleFormat validator (#1933).
- normalize_predictor bool|int -> int mapping and rejection branches.
- The opt-in perf gate (predictor=3 within 2x of predictor=2).

Boundary: GPU predictor files
(`test_predictor2_big_endian_gpu_1517.py`,
`test_predictor3_int_dtype_gpu_1933.py`) stay in the GPU cluster
issue #2438. GPU regressions co-located with CPU tests in the old
files move with this consolidation so behaviour-grouped coverage
stays together (matches PR #2451).

Audit: `xrspatial/geotiff/tests/CLUSTER_AUDIT_PREDICTOR.md` maps every
old `file::test` (with parametrise expansion) to the new
`file::test_id`. Deleted on the pre-merge commit before this lands on
main.

Closes #2427.
@github-actions github-actions Bot added the performance PR touches performance-sensitive code label May 26, 2026

@brendancol brendancol left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review: predictor cluster consolidation

Blockers

None.

Suggestions

None blocking.

Nits

  • xrspatial/geotiff/tests/unit/test_predictor.py:443-444: parametrise IDs use < / > for endianness. The issue body sketched id="predictor3[int16-big_endian]". The current [uint16->] form works but is harder to skim in pytest output; le / be would match the tiled / stripped choice you already used elsewhere in the file.
  • xrspatial/geotiff/tests/unit/test_predictor.py:494-502: nested-bracket IDs like pred2-libtiff[uint16] collect as [pred2-libtiff[uint16]]. Flat pred2-libtiff-uint16 reads better. Same for gpu-pred2[uint16], gpu-pred2-writer[uint16], gpu-pred2-encoder[uint16], pred2-writer[uint16].
  • xrspatial/geotiff/tests/unit/test_predictor.py:108 _da_xy: only called with 2D arrays here, but the 3D branch uses arr.shape[-2:] while the original helper used arr.shape[:2]. The original had a latent bug on 3D inputs (it would have read (band, y) as (h, w)); this quietly fixes it. A one-line comment would stop the next reader from "fixing" it back.
  • xrspatial/geotiff/tests/unit/test_predictor.py:443: imagecodecs_required lives mid-file. unit/test_codec_roundtrip.py declares all module-level skip markers near the top. Worth moving next to _gpu_only for consistency.
  • CLUSTER_AUDIT_PREDICTOR.md: the "+4 from filling out the BE x LE x dtype matrix" line is a touch off; the actual new IDs over the old file count are 3 (uint8-<, int8-<, int8->). The bottom summary still reconciles to +4 because the perf-skip case carries over. Deleted pre-merge anyway, so non-blocking.

What looks good

  • 76 old passes -> 80 new passes, same 1 skipped perf gate. Full geotiff suite still 5821 / 68 skipped / 2 xfailed locally.
  • GPU tests co-located with their CPU baselines, matching the photometric cluster (PR #2451). Easier to find a regression by behaviour than by hardware.
  • TN3 encoder (_tn3_encode_row) and the malformed predictor=3 + uint32 builder are kept verbatim, so the narrow #1247 / #1933 regressions still fire.
  • Section comments still cite issues #1313, #1220, #1247, #1479, #1933, so grep-by-issue keeps working.
  • imagecodecs guarded with pytest.importorskip so the file imports cleanly when the optional dep is missing.
  • Docstring back-references in test_kwarg_behaviour_2026_05_12_v2.py and integration/test_dask_pipeline.py updated to point at the new home.

Checklist

  • TN3 encoder/decoder math unchanged.
  • numpy + cupy + dask paths all exercised by the moved tests.
  • NaN handling: not in scope.
  • Edge cases: int8 wraparound, uneven tiles, 1024x1024 bit-level fidelity, single-band predictor=3, empty-tuple predictor.
  • Dask streaming write test moved over.
  • No premature materialization or extra copies.
  • Benchmark: N/A (tests-only).
  • README feature matrix: N/A (no API surface change).
  • Docstrings: section docstrings carry intent forward.

- Use `le` / `be` labels instead of `<` / `>` for endianness IDs.
- Flatten nested-bracket IDs (`pred2-libtiff[uint16]` ->
  `pred2-libtiff-uint16`, same for the other gpu / writer matrices)
  so pytest collection output reads as a single bracket per id.
- Move `imagecodecs_required` skip marker to the top of the file
  next to `_gpu_only` so all module-level gates live in one place.
- Annotate `_da_xy` about the trailing-axis indexing on 3D inputs
  so a future reader does not "fix" it back to the buggy
  pre-consolidation form.
- Reconcile the audit map and the +4 prose with the renamed IDs.

@brendancol brendancol left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up review (post-fix pass)

All five nits from the first pass are addressed in commit 4adb9d0:

  • Endianness IDs flipped to le / be (e.g. test_predictor2_round_trip_stripped[uint16-be], test_predictor3_round_trip_stripped[float32-le]).
  • Nested-bracket IDs flattened (pred2-libtiff-uint16, gpu-pred2-uint16, gpu-pred2-writer-uint16, gpu-pred2-encoder-uint16, pred2-writer-uint16).
  • imagecodecs_required moved to the top of the file next to _gpu_only.
  • _da_xy docstring now flags the trailing-axis indexing on 3D inputs so a future reader does not "fix" it back to the buggy pre-consolidation form.
  • Audit map and the +4 prose updated to reference the new IDs and explain the arithmetic.

pytest xrspatial/geotiff/tests/unit/test_predictor.py -q still reports 80 passed / 1 skipped after the renames. pytest xrspatial/geotiff/tests/unit/ -q reports 417 passed / 1 skipped (after the merge from main brought in the IFD cluster). Branch is merged with origin/main with no conflicts.

No remaining actionable findings; only the pre-merge audit deletion is outstanding.

The audit map served its purpose during review. Per the cluster
convention (see PR #2451 / #2450), the audit file is removed before
merge so it does not land on main.
@brendancol
brendancol merged commit a7a1bc8 into main May 26, 2026
7 checks passed
@brendancol
brendancol deleted the worktree-agent-a5f803a0349b781fd branch June 25, 2026 17:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance PR touches performance-sensitive code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Consolidation cluster 3: predictor (long-tail epic #2424)

1 participant