cold-store-reaper(listings): list_backups and s3_list_objects have no page/result bound — a fresh-token page stream loops forever, accumulating rows

Problem

Both listing loops in pkgs/cold-store-reaper/cold_store.py guard against a stuck cursor — a missing or repeated token raises — but neither bounds the number of pages or accumulated results, so a backend (or a hostile/buggy endpoint, or a proxy that mangles responses) that hands back a fresh token on every page makes the listing loop forever, growing the result list without limit:

Drive — ColdStore::list_backups (cold_store.py:1211, loop at :1223): termination is only nextPageToken falsy (:1251-1253) or repeated (:1254-1257, the cold-store-reaper: Drive list_backups never follows nextPageToken — prune-backups retention plans from a silently truncated listing #2426 fix). Every page appends to out (:1245-1250); there is no page cap and no result cap. list_backups feeds the list-backups CLI (:2425) and retention planning (:1344).
S3 — ColdStore::s3_list_objects (cold_store.py:1805, loop at :1817): the docstring states the progress guard — a truncated page must return a usable, advancing NextContinuationToken, else raise (:1810-1816, implemented :1827-1834, the cold-store-reaper: s3_list_objects loops forever on a truncated page with a missing, empty, or repeated continuation token #1632 fix). But a page stream that is always IsTruncated=true with a new token each time satisfies that guard on every iteration: out.extend(page) (:1824) grows forever and the function never returns.

The existing guards answer "is the cursor advancing?" — nothing answers "has this listing gone on implausibly long?". A real Drive folder of backups or an R2 bucket holds a bounded, knowable number of objects; an endless fresh-token stream is a malfunction, and the correct response is the same one the sibling guards already chose: raise a ColdStoreError naming the listing, not hang.

Evidence (verified this run, laptop @ origin/main `fa2bb52`)

Probe: ~/git/fleet/probe-coldstore-pages/probe.py — the real cold_store module imported from the laptop primary, transports stubbed (http_request for Drive, _s3_request for S3), stub gives up with a sentinel after 5,000 calls:

Drive control, finite: 2 pages (token then none) → returned, 2 calls, 2 rows. ✅
Drive control, repeated token: same token every page → ColdStoreError ("files.list for backups repeated nextPageToken"), 2 calls. ✅
Drive defect: every page returns 1 file and a brand-new nextPageToken → still looping at 5,001 calls; the stub's sentinel, not the listing, ended it.
S3 control, finite: truncated page + token, then untruncated page → returned, 2 calls, 2 objects. ✅
S3 control, repeated token: IsTruncated=true with the same token → ColdStoreError ("non-advancing NextContinuationToken"), 2 calls. ✅
S3 defect: every page IsTruncated=true with a brand-new token → still looping at 5,001 calls; only the stub's sentinel ended it.

Probe verdict line: "PROBE PASS: both listings loop unbounded on fresh-token streams".

Why it matters

These listings are the reaper's view of what exists: list_backups drives retention/prune planning and the operator list-backups command; s3_list_objects enumerates buckets for the same lifecycle work. A wedged endpoint, a token-minting bug server-side, or a response-rewriting proxy turns either call into a permanent hang with unbounded memory growth (the accumulated list) instead of a diagnosable error — in a tool whose whole job is unattended storage hygiene, a silent hang is the failure mode that pages nobody and prunes nothing.

Distinct from closed #2426 (Drive listing did not follow nextPageToken at all — now it does) and closed #1632 (S3 missing/empty/repeated token — now guarded): this is the remaining axis, volume — page/result count — which neither fix bounded. Sibling in spirit to open #6572 (github-backup drain pagination, different codebase and endpoint).

Acceptance criteria

Both list_backups (Drive) and s3_list_objects (S3) terminate with a ColdStoreError naming the listing when a page stream exceeds a generous page and/or result bound sized well above any real backup folder / bucket (the exact bound is the fixer's choice; it must be unreachable by a legitimate listing).
Regression tests: a stubbed fresh-token page stream raises the new error for both listings (not a hang); the existing finite multi-page controls and the repeated/missing-token guards keep passing.
Existing cold-store-reaper tests stay green.

CODE-2012

🤖 Agent note · filed

Agent: new-issue-creator-code-a (new-issue-creator job, lane A)
Session / ID: job new-issue-creator-code-a, run 2026-11-06 12:35 CST
Found via: filed this run
Task: Filed a verified unbounded-pagination defect: both cold_store listing loops (Drive list_backups, S3 s3_list_objects) guard missing/repeated tokens but have no page/result bound, so a fresh-token page stream loops forever accumulating rows. Verified on the laptop @ origin/main fa2bb52 with probe ~/git/fleet/probe-coldstore-pages (real module, stubbed transports): finite controls returned in 2 calls, repeated-token controls raised ColdStoreError, fresh-token streams were still looping at 5,001 calls for both listings.
Expectation: effort/s — triage confirms, then a fix adds a generous page/result bound to both loops with stubbed fresh-token regression tests.
Links: issue body (this issue); packet packet-triage-6602.json (enqueue attempted this run; queue backend is in a PostgREST global-timeout regime — see backlog)

🤖 Agent note · design-architect pass 63 — adopted as a phase of design #6622

Design: Design: complete, bounded, or say so — one pagination contract for the repo's paged reads #6622 — complete, bounded, or say so (one pagination contract for the repo's paged reads). Doc: files/designs/code-pagination-contract.md in the keep-the-four-repos-healthy goal workspace. Audit at origin/main 075b2de90
This issue is Phase 2 (volume bounds on fresh-cursor streams), with the other Phase 2 issues (github-backup(drain): queue_events pagination has no page/event bound — a Worker page stream with fresh cursors and list_complete=false loops forever, accumulating events and never acking #6572, github-backup(ghb_common): inventory_owner paginates forever when a page says hasNextPage with a null/empty endCursor — the reconcile sweep hangs re-fetching page 1 instead of raising #6532). Linked as a sub-issue of Design: complete, bounded, or say so — one pagination contract for the repo's paged reads #6622 (verified in its sub_issues list). Independent of Phase 1 in code
Decision recorded on Design: complete, bounded, or say so — one pagination contract for the repo's paged reads #6622 that binds this phase: every page loop carries a page-count and/or accumulated-result bound, sized generously above any legitimate listing, whose breach is an error naming the listing — the bound's exact value stays with this phase's fixer as a named constant; a fresh-token stream ending in a hang is the one forbidden outcome
For triage: do NOT re-file, re-scope or split this issue as a standalone — record the phase structure instead. No new packet from the architect: this issue already carries its own queue lifecycle

Picked up by repo-worker-code-a — working this now. (Fleet queue fallback mode: the queue backend is down, so this issue's comments carry the claim. The claim stays live while repo-worker-code-a posts progress markers; after 30 minutes of silence any other worker may pick this up.)

🤖 Agent note · picked up

Agent: queue worker (repo-worker-code), lane code-a — FALLBACK MODE (queue backend down; fallback claim marker on this issue is the claim of record)
Session / ID: cron repo-queue-worker-shift run 2026-11-06 19:20 CST; fallback worker id repo-worker-code-a (issue cold-store-reaper(listings): list_backups and s3_list_objects have no page/result bound — a fresh-token page stream loops forever, accumulating rows #6602 is the task in fallback mode)
Found via: fallback scan — my own routed fix vehicle from this shift's triage of this issue
Task: small-fix — add a generous page/result bound to both list_backups (Drive) and s3_list_objects (S3) in pkgs/cold-store-reaper/cold_store.py, raising ColdStoreError naming the listing when exceeded; stubbed fresh-token regression tests for both loops
Expectation: s — done = PR opened, laptop tests green, merged under the standing directive, issue closed via the PR
Links: triage verdict on this issue (comment 6701355012) · branch fb-6602-cold-store-page-bound

Progress from repo-worker-code-a: PR #6717 opened for the #6602 listing bounds (LIST_MAX_PAGES/LIST_MAX_RESULTS in both loops; laptop full suite 115 tests OK incl. 3 new fresh-token regression tests). Watching Codex, then merge under the standing directive.

Merge pull request #6717 from developing-today/fb-6602-cold-store-pag…

🤖 Agent note · done

Agent: queue worker (repo-worker-code), lane code-a — FALLBACK MODE (queue backend down; fallback claim markers on this issue are the claim of record)
Session / ID: cron repo-queue-worker-shift run 2026-11-06 19:20 CST; fallback worker id repo-worker-code-a (issue cold-store-reaper(listings): list_backups and s3_list_objects have no page/result bound — a fresh-token page stream loops forever, accumulating rows #6602 is the task in fallback mode)
Found via: fallback scan — my own routed fix vehicle from this shift's triage of this issue
Task: small-fix — page/result bounds for the Drive and S3 listing loops
Result: PR fix(cold-store-reaper): bound the Drive and S3 listing loops (#6602) #6717 merged (ff160b9964): LIST_MAX_PAGES (10_000) and LIST_MAX_RESULTS (1_000_000) checked per page in both list_backups and s3_list_objects, raising ColdStoreError naming the listing. Laptop: full tests.test_cold_store suite 115 tests OK including 3 new fresh-token regression tests (Drive page bound, Drive result bound, S3 page bound — each trips after exactly the capped fetches). Codex review: zero findings. PR CI stayed QUEUED (runner backlog); merged under the standing directive on laptop verification. Issue auto-closed by the PR.

Released by repo-worker-code-a (outcome: done). Fixed and merged via PR #6717 (merge ff160b9): LIST_MAX_PAGES/LIST_MAX_RESULTS bounds in both listing loops, ColdStoreError naming the listing; laptop suite 115 tests OK incl. 3 new fresh-token regression tests; Codex zero findings; issue auto-closed by the PR.

Navigation Menu

Repository navigation

cold-store-reaper(listings): list_backups and s3_list_objects have no page/result bound — a fresh-token page stream loops forever, accumulating rows #6602

Problem

Evidence (verified this run, laptop @ origin/main `fa2bb52`)

Why it matters

Acceptance criteria

14 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

cold-store-reaper(listings): list_backups and s3_list_objects have no page/result bound — a fresh-token page stream loops forever, accumulating rows #6602

Description

Problem

Evidence (verified this run, laptop @ origin/main fa2bb52)

Why it matters

Acceptance criteria

Activity

developing-today commented on Nov 6, 2026

dezren39 commented on Nov 6, 2026

dezren39 commented on Nov 6, 2026

dezren39 commented on Nov 7, 2026

14 remaining items

dezren39 commented on Nov 7, 2026

dezren39 commented on Nov 7, 2026

dezren39 commented on Nov 7, 2026

dezren39 commented on Nov 7, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Evidence (verified this run, laptop @ origin/main `fa2bb52`)