Skip to content

id: design: deterministic testing over time — simulated clock, replay, seeded fuzz corpus, soak runs #5779

Open

Description

@dezren39

Summary

Design ID's deterministic testing infrastructure: the machinery that makes the system's behaviour reproducible — a simulated clock, deterministic replay of recorded world event streams, a managed seeded fuzz corpus, and long-horizon soak runs with invariant checkers. This is testing infrastructure as its own design line, distinct from any one feature's tests.

Motivation

ID is a journal/event-sourced system: correctness lives in replay, reconstruction, and projection. The pieces that exist are scoped elsewhere — #558's deterministic fuzz harness belongs to the authoring dev loop; #544 is a per-world replay self-check (id world verify); #5685 designs the DoS suite and the performance benchmark programme (regression gates on speed and abuse resistance — it says nothing about determinism). The mcpx deterministic-testing work (#1963, #2641) is mcpx-scoped. Nothing designs, for ID itself: how determinism is achieved (where nondeterminism can enter), how recorded reality becomes a permanent regression asset, or how the system is exercised over time — days of simulated operation, not single runs.

The design's own rules are the natural oracles: "reconstruction is admission" (#2898), lease-epoch monotonicity and single-writer fencing (#2819), writer-set projection purity (#5545 — projection is a pure function of its tuple, so any replay divergence is by definition a bug).

Sketch

  • Simulated clock: logical time injected at the core seams; wall-clock reads, RNG, hash-map iteration order, and thread scheduling audited as nondeterminism sources, with the audit itself a deliverable. Where the clock seam lives across the crate split (pkgs/id: split the monolith crate into sub-crates along module seams (prioritized by Drew 2026-11-01) #3447) is part of the design.
  • Deterministic replay: capture world event streams (journals plus input frames) into replayable fixtures; replay must reproduce the journal hash bit-exact; divergence is a bug report with a minimal repro, not a flaky test.
  • Seeded fuzz corpus: corpora as versioned artefacts — seeds that found bugs are pinned forever, minimisation is automated, corpus growth is reviewed like code.
  • Soak runs: long-horizon runs compressing days of simulated time, with invariant checkers running continuously over live state (not just at checkpoints): admission invariants, lease fencing, projection purity, discovery bounds.
  • CI shape: a determinism gate (same input → same journal hash) on the fast tier; replay + soak on a slower cadence, sized with design(id): DoS suite & performance benchmark programme — adversarial CI coverage and per-interface regression gates #5685's tiering so the two programmes share load machinery instead of duplicating it.

Open questions

Relations

Parent: epic #2056. Relations: #558 (dev-loop fuzz harness — a consumer), #544 (replay self-check — a consumer), #5685 (DoS + benchmark programme — sibling infra, shared machinery), #2898, #2819, #5545 (oracle rules), #3447 (crate seams). Filed in the 2026-11-04 gap pass (WS-F) per Drew's order that deterministic testing gets its own design issue.

Activity

  1. added
    app/idPackage/app: id (pkgs/ or apps/ in this repo)
    on Nov 5, 2026
  2. developing-today commented on Nov 5, 2026

    @developing-today
    Contributor
  3. added
    queue/queuedHas a queued task in the Supabase agent queue
    task/triageQueue packet kind: verify, reproduce, and route this item
    on Nov 5, 2026
  4. added
    work/assignedQueue worker lifecycle state
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    status/parkedDeliberately paused with a recorded resume condition
    and removed
    queue/queuedHas a queued task in the Supabase agent queue
    on Nov 5, 2026
  5. developing-today commented on Nov 5, 2026

    @developing-today
    Contributor

    🤖 Agent note · handed off

  6. developing-today commented on Nov 5, 2026

    @developing-today
    Contributor

    🤖 Agent note · handed off / parked

    • Agent: queue worker (repo-worker-code), lane code-l
    • Session / ID: queue task 10183
    • Found via: atomic queue claim (lane code-l); execution-lane assignment returned opencode, so per runbook §1b this task is handed off to the laptop opencode claimant — not worked directly by this lane.
    • Task: triage id: design: deterministic testing over time — simulated clock, replay, seeded fuzz corpus, soak runs #5779 — id: design: deterministic testing over time (simulated clock, replay, seeded fuzz corpus)
    • Revalidation (fresh, 2026-11-05T14:09:12Z, via developing.today GitHub App token): issue OPEN, no hard-stop labels (state/labels: OPEN task/triage,status/parked,queue/parked,app/id,work/assigned).
    • Action: task 10183 parked as handed off; the claimant launches the bound opencode session to do the work.
  7. dezren39 commented on Nov 5, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · handed off

  8. dezren39 commented on Nov 5, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · handed off

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

app/idPackage/app: id (pkgs/ or apps/ in this repo)queue/parkedQueue task is parked (waiting on CI, time, or an event)status/parkedDeliberately paused with a recorded resume conditiontask/triageQueue packet kind: verify, reproduce, and route this itemwork/assignedQueue worker lifecycle state

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions