fix(id): name the hanging group in nixos-integration VM test (#1372) by dezren39 · Pull Request #4913 · developing-today/code

dezren39 · 2026-11-03T13:39:44Z

What

pkgs/id/nix/tests/integration-test.nix ran the entire cli_integration
binary as one server.succeed() with --test-threads=2 --nocapture --format=terse, so a hang produced only timeout reached; test terminating...
with no test name or wait target (#1372, recurrence of #896's acceptance).

This makes the harness legible without changing what is covered:

Four named groups — fast_unit (49 tests), serve (43), tag (29),
harness_unit (5) — each run under its own GNU timeout deadline
(900/2200/400/300 s) instead of one 3600 s globalTimeout.
The wait target is logged before every group: group name, libtest
filters, effective deadline, and remaining suite budget.
--format=pretty replaces --format=terse, so completed tests stream
into the log and libtest prints test <name> has been running for over 60 seconds for anything still running.
A 3400 s suite budget clamps each group's deadline to what is left, so a
hang always fails with a named group inside the budget rather than at the
driver's globalTimeout = 3600 (which stays as the backstop).
Every group still runs after a failure, and the run ends with a summary
naming all failed groups — a failing serve no longer hides tag or
harness_unit results.
A coverage guard runs TEST_BIN --list first and proves the group
filters partition every test exactly once. A new test module that is not in
GROUPS fails immediately with the uncovered names instead of silently
being dropped.

Validation (real VM, 2026-11-03)

nix build .#checks.x86_64-linux.nixos-integration on the branch, reusing the
branch's built integrationTestRunner and id binary. The stock build is
currently blocked before the VM by the pre-existing #4850 checkPhase failure,
so only that unrelated id-web checkPhase was skipped; the harness and all
126 tests ran unchanged.

[id-integration] 126 tests in 4 named groups: fast_unit=49, serve=43, tag=29, harness_unit=5
[id-integration] waiting on group 'fast_unit' (...); per-group deadline 900s, 3400s of suite budget left
[id-integration] group 'fast_unit' finished in 7.4s
[id-integration] waiting on group 'serve' (filters: serve_tests::); per-group deadline 2200s, 3384s of suite budget left
test result: FAILED. 40 passed; 3 failed; ... finished in 17.06s
[id-integration] group 'serve' failed or exceeded its 2200s deadline; filters: serve_tests::; continuing
[id-integration] group 'tag' finished in 1.2s
[id-integration] group 'harness_unit' finished in 4.3s
AssertionError: [id-integration] failed groups: serve

The three serve failures are pre-existing and now name themselves:
test_world_join_via_ssh_carries_the_frame_protocol,
test_world_ssh_tunnel_carries_join_via_ssh, and
test_world_module_shared_over_iroh_and_web_presentations, all rooted in the
id binary being built without the ssh feature — filed as #4960. Before
this change that failure arrived as a bare terse log.

A deliberate short-deadline run (same harness, serve deadline 5 s) produced:

[id-integration] waiting on group 'serve' (filters: serve_tests::); per-group deadline 5s, 3392s of suite budget left
AssertionError: [id-integration] group 'serve' failed or exceeded its 5s deadline; filters: serve_tests::

Other checks: nix eval of the check derivation, python3 -m py_compile of
the emitted testScript, nixfmt clean.

Remaining blockers for a green stock check (pre-existing, not this PR)

id(chain): ChainExt literals in pkgs/id tests missing the #3794 checkpoint field — cargo check --all-targets fails on main #4850 — id-web checkPhase fails on main (ChainExt literals missing the
id: protocol v1 phase 1 — session header + segment checkpoint event kind (#2036) #3794 checkpoint field).
id(ci): nixos-integration's id binary lacks the ssh feature, so the SSH world tests fail #4960 — the VM test's id binary is built without ssh, so the SSH world
tests cannot pass.

This PR is reporting-only; the suite's coverage is unchanged.

Refs #1372. Scope deliberately excludes chasing which test hangs and editing
the integration tests themselves.

The whole cli_integration binary ran in one server.succeed() with --format=terse, so a hang produced only "timeout reached; test terminating..." with no test name or wait target. Run the suite as four named groups (fast_unit, serve, tag, harness_unit), each under its own timeout deadline and with the pretty libtest format. A hang now names the group and filters before the per-group deadline fires, and libtest names the individual tests still running after 60 s. A coverage check proves the filters partition every test in the binary before anything runs, so no test is dropped. A 3400 s suite budget keeps every failure below the 3600 s globalTimeout backstop. Coverage and the deadline failure path were verified against the locally built test binary; the full VM run is the remaining validation.

coderabbitai · 2026-11-03T13:39:54Z

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 21 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

Source mode: Auto
Repository: Unavailable
Branch: Unavailable
File: No configuration file from this repository applied
Commit: Unavailable
Layers: defaults
Review profile: CHILL
Plan: Advanced
Run ID: 2ed8ed82-b3fa-44b9-b1a3-f6cd70577e0a

Commits

Reviewing files that changed from the base of the PR and between a75abab and 6305f6a.

Files selected for processing: 1

pkgs/id/nix/tests/integration-test.nix

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

CodeRabbit Verify

Run CodeRabbit Verify to test the changes in this pull request.

chatgpt-codex-connector · 2026-11-03T13:39:55Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	⚠️ Failed 2026-11-03T13:39:49.762560Z We couldn’t complete this review. Comment `@codex review` to try again.	`fe7b78f`	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

chatgpt-codex-connector · 2026-11-03T13:39:58Z

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

…96-1372-legible-vm-timeout

…s rc)

…96-1372-legible-vm-timeout

dezren39 requested a review from a team as a code owner November 3, 2026 13:39

dezren39 mentioned this pull request Nov 3, 2026

id(ci): nixos-integration VM test still times out silently — #896 acceptance recurrence (nightly 40457980141) #1372

Open

dezren39 added 3 commits November 3, 2026 07:46

Merge remote-tracking branch 'refs/remotes/origin/main' into queue/77…

f8bc1a2

…96-1372-legible-vm-timeout

fix(id): use succeed for group runs (execute(check_return=False) hide…

3e64b82

…s rc)

fix(id): run every group before failing, then name all failed groups

cc96cc4

dezren39 mentioned this pull request Nov 3, 2026

id(ci): nixos-integration's id binary lacks the ssh feature, so the SSH world tests fail #4960

Closed

Merge remote-tracking branch 'refs/remotes/origin/main' into queue/77…

6305f6a

…96-1372-legible-vm-timeout

dezren39 merged commit a1618c2 into main Nov 3, 2026
2 checks passed

dezren39 deleted the queue/7796-1372-legible-vm-timeout branch November 3, 2026 14:45

Navigation Menu

Repository navigation

fix(id): name the hanging group in nixos-integration VM test (#1372) - #4913

fix(id): name the hanging group in nixos-integration VM test (#1372)#4913
dezren39 merged 5 commits into
mainfrom
queue/7796-1372-legible-vm-timeout

dezren39 commented Nov 3, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Nov 3, 2026 •

edited

Loading

Review limit reached

Uh oh!

chatgpt-codex-connector Bot commented Nov 3, 2026

Uh oh!

chatgpt-codex-connector Bot commented Nov 3, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

dezren39 commented Nov 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Validation (real VM, 2026-11-03)

Remaining blockers for a green stock check (pre-existing, not this PR)

Uh oh!

coderabbitai Bot commented Nov 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review limit reached

CodeRabbit Verify

Uh oh!

chatgpt-codex-connector Bot commented Nov 3, 2026

Codex Review Summary

Uh oh!

chatgpt-codex-connector Bot commented Nov 3, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

dezren39 commented Nov 3, 2026 •

edited

Loading

coderabbitai Bot commented Nov 3, 2026 •

edited

Loading