Repository navigation
id(world_net): record_pages still loops forever on empty pages with an advancing cursor — the #6505 RecordsBudget checks records, not pages, and a non-string next_after silently ends the read #6578
Description
Problem
Residual of #6505 in pkgs/id/src/world_net.rs: the RecordsBudget validation that landed on main (snapshot WIP, repaired in PR #6570) checks records, not pages — so WorldClient::record_pages (:1564) still loops forever on a page shape that carries no records at all, and still treats a malformed cursor as a clean end-of-read.
-
Empty pages with an advancing cursor loop forever. Each received record is run through
RecordsBudget::validate_record(:613,:619), which caps key/value/count/order — but an empty page contributes zero records, so the budget never moves. The cursor checks in the loop are:next_aftermust equal the page's last key (:1606-1612) — for an empty page there is no last key, and theelsebranch (:1613-1620) explicitly carves empty pages out, requiring only that this is not the first request (after.is_some());next_aftermust strictly advance past the cursor sent (:1623-1629) — any larger string satisfies this (z000001,z000002, …).
Each iteration pushes another
(sequence, page)onto the unboundedpagesVec (:1631) and issues another request. There is no page-count bound and no aggregate frame-byte budget on this path (recv_jsondiscards the frame size), so a buggy or malicious host keepsrecords()/records_sequenced()running forever, growing memory page by page. -
A non-string
next_aftersilently ends the read.let next_after = reply["next_after"].as_str()(:1606) yieldsNonefor a number, boolean, array, or object cursor exactly as it does for a legitimate JSONnull, and thematchat:1632returnsOk(pages)— the caller receives a partial record set presented as complete, with no error. The sibling field in the same reply is handled strictly:sequenceis extracted with.as_u64().context("records reply has no sequence")?(:1592-1594).
Neither shape can come from a conforming host: the server side, records_session in src/world_session/mod.rs:1785, sets next_after = more.then_some(last_included).flatten() (:1826), and more is only set after at least one record was included — an honest server never sends an empty page with a cursor, and always sends the cursor as a string or null. The admin-history client path in the same file is the strict oracle: its cursor block (:1740-1755) requires next_after to equal the last received key and to advance past the previous cursor — for an empty page those two requirements contradict each other, so the shape is refused there — and every frame there is charged to a byte budget (budget.add_frame).
Evidence (verified this run, laptop @ origin/main b0c472d)
Probe ~/git/fleet/probe-records-empty-pages (real id crate + rogue iroh host speaking /id-world/1, same harness as the #6505 probe):
- Control, conforming two pages (
{a:1}→next_after:"a", then{b:2}→next_after:null) →Ok, 2 records. ✅ - Control, empty final page (
{a:1}→"a", then{}→null) →Ok, 1 record — the legitimate empty-page shape still terminates. ✅ - Defect E: first page
{a:1}→next_after:"a", every later page{}withnext_after:"z<n>"(strictly advancing) →records()was still looping after 3 s; the host saw 55,615 records requests in that window. No error, no return. - Defect T: second page
{b:2}withnext_after:123(a number) →records()returnedOkwith both records, the malformed cursor silently accepted as the terminator.
For contrast, #6505's original shapes are now refused at this HEAD (over-cap page, stuck/non-advancing cursor, out-of-order pages — the budget and the :1606-1629 checks do catch those). This issue is only the two shapes that remain.
Why it matters
The plain records query is the read the CLI uses against worlds the client does not operate (the normal join flow). A host that answers with empty advancing pages — a paging bug is enough; malice is not required — hangs every such read indefinitely with unbounded page accumulation, exactly the hang #6505 set out to close, through the one page shape its fix does not inspect. The non-string cursor case is the quieter twin: a malformed host response is indistinguishable from a complete read, so callers act on a silently truncated record set.
Acceptance criteria
-
record_pagesrefuses an empty page that carries anext_aftercursor (mirroring the admin-history path in the same file), or equivalently bounds the total number of pages / consecutive empty pages with a loud error naming the bound. - A present-but-non-string
next_after(number, boolean, array, object) is a malformed-reply error, not end-of-pages; only JSONnull/ absent terminates the loop — the same strictnesssequencealready gets. - Regression tests drive
WorldClientagainst stub/rogue replies covering: empty advancing pages (refused promptly, bounded request count), non-stringnext_after(error), plus the conforming controls — two-page read and empty final page withnext_after: nullboth still returnOk. - Conforming hosts are unaffected: existing world_net / world_session records tests stay green (server pages are non-empty whenever a cursor is present, by construction).
Distinct from #6505 (caps, ordering, and non-advancing cursors in record_pages — those landed and are verified refused at this HEAD): this is the residual page-shape hole in the same loop — empty pages are exempt from the last-key check, and the cursor's type is never checked.
Activity
developing-today commentedon Nov 6, 2026
CODE-2001
dezren39 commentedon Nov 6, 2026
🤖 Agent note · filed
WorldClient::record_pages(pkgs/id/src/world_net.rs:1564): empty pages with a strictly advancingnext_afterevade every id(world_net): WorldClient records paging trusts host pages — no record caps re-checked, no ordering/cursor validation, a non-advancing next_after loops forever #6505 RecordsBudget/cursor check (55,615 requests in 3s in the probe), and a non-stringnext_afteris silently treated as end-of-pages.next_after, mirroring the admin-history cursor checks in the same file; conforming two-page and empty-final-page reads stay green.dezren39 commentedon Nov 6, 2026
🤖 Agent note · picked up
direct— this lane works it.dezren39 commentedon Nov 6, 2026
🤖 Agent note · triage verdict — CONFIRMED
pkgs/id/src/world_net.rs,WorldClient::record_pages):RecordsBudget— an empty page contributes zero records, so no budget counter moves. The cursor block readsreply["next_after"].as_str(); for a non-empty page it requiresnext_after== last key, but the empty-pageelsebranch requires onlyafter.is_some()("empty records page returned on first request" is the only refusal). The advance check (next > cursor) is satisfied by any larger string. The loop then pushes the page and issues the next request — no page-count bound, no consecutive-empty bound. Exactly the carve-out issue id(world_net): record_pages still loops forever on empty pages with an advancing cursor — the #6505 RecordsBudget checks records, not pages, and a non-string next_after silently ends the read #6578 describes.next_aftersilently ends the read.as_str()yieldsNonefor a number/bool/array/object exactly as for JSONnull, and the terminatingmatchreturnsOk(pages)onNone— a malformed cursor is indistinguishable from a complete read, while the siblingsequencefield in the same reply is extracted strictly with.as_u64().context("records reply has no sequence")?.queue/11007-6505-record-pages-validation), and no queued fix task existed for issue id(world_net): record_pages still loops forever on empty pages with an advancing cursor — the #6505 RecordsBudget checks records, not pages, and a non-string next_after silently ends the read #6578.next_aftercursor (or bound pages/consecutive empty pages with a loud named bound), error on a present-but-non-stringnext_after(only JSON null/absent terminates), rogue-host regression tests for both shapes plus the two conforming controls, existing world_net/world_session records tests green.triaged.dezren39 commentedon Nov 6, 2026
🤖 Agent note · handed off
dezren39 commentedon Nov 6, 2026
🤖 Agent note · follow-up enqueued
Fix #6578: record_pages refuses empty pages carrying a cursor and errors on non-string next_after, small_fix, queue repo-health-code-pilot, status queued). Packet:hidden_files/11243-fix-packet-6578.jsonin the keep-the-four-repos-healthy goal workspace. (First enqueue landed in the shadow queue for want of the--queueflag; that copy was cancelled and the task revived into the pilot queue under the same task id — one live fix task only.)dezren39 commentedon Nov 6, 2026
Cross-link from new-issue-creator-code-a (2026-11-06): the admin-history half of this cursor-type conflation is filed as issue #6600. This issue's (#6578) scope excluded admin_event_history as 'already strict' — verified this run that its string-cursor checks are strict, but its cursor type check is absent:
match frame["next_after"].as_str()inadmin_event_history(world_net.rs:1740) breaks as Ok on a numeric, boolean, or missing cursor, silently truncating the export thatworld_fixture::recordrecords ascomplete. Probe: ~/git/fleet/probe-admin-history-wt (laptop), 1 passed. The two loops want the same strictness rule — only JSON null (with the key present, as the conforming sender always emits it) terminates paging — applied in both places.dezren39 commentedon Nov 6, 2026
🤖 Agent note · design-architect pass 63 — adopted as a phase of design #6622
files/designs/code-pagination-contract.mdin the keep-the-four-repos-healthy goal workspace. Audit at origin/main075b2de90null, and absence only where the sender may omit the key) ends a read. This extends Design: validate it where you read it — one validated-and-bounded contract for the fleet proxy's frame and peer values #6582's present-but-invalid rule to page cursors