Description
Summary
Design ID's deterministic testing infrastructure: the machinery that makes the system's behaviour reproducible — a simulated clock, deterministic replay of recorded world event streams, a managed seeded fuzz corpus, and long-horizon soak runs with invariant checkers. This is testing infrastructure as its own design line, distinct from any one feature's tests.
Motivation
ID is a journal/event-sourced system: correctness lives in replay, reconstruction, and projection. The pieces that exist are scoped elsewhere — #558's deterministic fuzz harness belongs to the authoring dev loop; #544 is a per-world replay self-check (id world verify); #5685 designs the DoS suite and the performance benchmark programme (regression gates on speed and abuse resistance — it says nothing about determinism). The mcpx deterministic-testing work (#1963, #2641) is mcpx-scoped. Nothing designs, for ID itself: how determinism is achieved (where nondeterminism can enter), how recorded reality becomes a permanent regression asset, or how the system is exercised over time — days of simulated operation, not single runs.
The design's own rules are the natural oracles: "reconstruction is admission" (#2898), lease-epoch monotonicity and single-writer fencing (#2819), writer-set projection purity (#5545 — projection is a pure function of its tuple, so any replay divergence is by definition a bug).
Sketch
- Simulated clock: logical time injected at the core seams; wall-clock reads, RNG, hash-map iteration order, and thread scheduling audited as nondeterminism sources, with the audit itself a deliverable. Where the clock seam lives across the crate split (pkgs/id: split the monolith crate into sub-crates along module seams (prioritized by Drew 2026-11-01) #3447) is part of the design.
- Deterministic replay: capture world event streams (journals plus input frames) into replayable fixtures; replay must reproduce the journal hash bit-exact; divergence is a bug report with a minimal repro, not a flaky test.
- Seeded fuzz corpus: corpora as versioned artefacts — seeds that found bugs are pinned forever, minimisation is automated, corpus growth is reviewed like code.
- Soak runs: long-horizon runs compressing days of simulated time, with invariant checkers running continuously over live state (not just at checkpoints): admission invariants, lease fencing, projection purity, discovery bounds.
- CI shape: a determinism gate (same input → same journal hash) on the fast tier; replay + soak on a slower cadence, sized with design(id): DoS suite & performance benchmark programme — adversarial CI coverage and per-interface regression gates #5685's tiering so the two programmes share load machinery instead of duplicating it.
Open questions
- Corpus storage and size discipline: in-repo, in object storage, or generated-from-seeds only?
- Soak cadence and budget: nightly vs weekly; what simulated-time-per-real-hour is achievable?
- How much of the mcpx determinism pattern (mcpx testing: deterministic execution as a long-term aspiration (retrofit vs Rust rewrite) #1963/mcpx testing: one deterministic-hypervisor (Antithesis) experiment on the pkgs/mcpx container image #2641) is shared doctrine vs per-system machinery?
- Does the simulated clock extend to network/transport simulation (partitions, latency) or stay at the world layer for v1?
Relations
Parent: epic #2056. Relations: #558 (dev-loop fuzz harness — a consumer), #544 (replay self-check — a consumer), #5685 (DoS + benchmark programme — sibling infra, shared machinery), #2898, #2819, #5545 (oracle rules), #3447 (crate seams). Filed in the 2026-11-04 gap pass (WS-F) per Drew's order that deterministic testing gets its own design issue.
Activity
developing-today commentedon Nov 5, 2026
CODE-1463
developing-today commentedon Nov 5, 2026
🤖 Agent note · handed off
developing-today commentedon Nov 5, 2026
🤖 Agent note · handed off / parked
repo-worker-code), lane code-lopencode, so per runbook §1b this task is handed off to the laptop opencode claimant — not worked directly by this lane.dezren39 commentedon Nov 5, 2026
🤖 Agent note · handed off
dezren39 commentedon Nov 5, 2026
🤖 Agent note · handed off
opencode, so this task is handed off to the opencode claimant — no direct triage work by this lane. Fresh check on claim: issue id: design: deterministic testing over time — simulated clock, replay, seeded fuzz corpus, soak runs #5779 OPEN, no hard-stop labels.