Skip to content

fix: eliminate flaky CI races - #43522

Merged
kitlangton merged 4 commits into
v2from
fix-flaky-tests
Aug 19, 2026
Merged

kitlangton merged 4 commits into
v2from
fix-flaky-tests

Conversation

@kitlangton

@kitlangton kitlangton commented Aug 19, 2026 •

Copy link
Copy Markdown
Contributor

What

Eliminate the reproducible CI races found across the last two days of V2 test runs and a full local stress pass.

  • prevent one in-place TUI plugin save from activating multiple transient generations
  • isolate CLI subprocess tests from the developer's real config, database, service port, and process lifetime
  • remove low-value timing, fixture-smoke, and duplicate protocol coverage
  • harden process, shell, worktree, and ACP cleanup tests at their actual synchronization boundaries

Fixes #37321.

Before / After

Before: Local plugin reload hashed a file and imported it in two separate reads. An in-place save could expose transient bytes to the hash and completed bytes to the import, assigning one module several generations and repeatedly cycling setup/cleanup. Other tests treated path existence as readiness, inherited ambient V2 state, relied on fixed wall-clock sleeps, force-killed subprocesses before deleting their directories, or put real Git/process integration work under Bun's five-second default.

After: Plugin setup runs only when the generation still matches after import. Subprocesses own isolated state and shut down normally before cleanup. Readiness waits for actual content, and only the integration cases proven too expensive on loaded Windows runners receive explicit 15-second budgets. Tests that only exercised timers, fixtures, test literals, or duplicate protocol paths are removed.

How

  • packages/tui/src/plugin/context.tsx verifies the content generation after dynamic import and re-resolves when an in-place write crossed the hash/import boundary.
  • packages/tui/test/plugin-hot-reload.test.tsx deterministically pauses an import, rewrites the entrypoint, and proves only the final generation activates.
  • packages/cli/src/services/service-registration.ts extracts the unchanged managed-service lease so stale registration replacement can be tested without booting a full CLI process.
  • CLI subprocess fixtures use isolated HOME/config/database paths, select an unused managed-service port, close ACP over stdin EOF before forced termination, and give Windows directory locks a bounded retry window.
  • Core readiness and integration tests wait for non-empty readiness files and use the same explicit timeout budget already used by neighboring shell/Git integration tests.
  • Theme API canaries remain in typecheck-only coverage while stale test-only mirrors, fixture smoke tests, duplicate ACP checks, and host-performance assertions are removed.

Scope

Testing

  • bun run test in packages/tui: 728 pass, 5 skip
  • bun run test in packages/core: 1881 pass, 16 skip
  • bun run test in packages/cli: 205 pass
  • bun typecheck in packages/tui, packages/core, and packages/cli
  • pre-push repository typecheck: 33 packages pass
  • plugin recovery: 200 repeated passes
  • normal and symlinked plugin reload: 100 repeated passes each
  • deterministic hash/import generation regression: 20 repeated passes
  • process readiness: 500 repeated passes after a 1/100 baseline reproduction
  • stale worktree, shell timeout, service lease, standalone, raw ACP, and ACP cleanup targeted stress loops: all pass

Flow

flowchart LR
  A[Watcher observes local plugin] --> B[Hash source generation]
  B --> C[Import generation URL]
  C --> D{Source generation unchanged?}
  D -- no --> B
  D -- yes --> E[Validate plugin]
  E --> F[Activate exactly once]
Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant