CI: unit (linux) turbo run segfaulted mid-suite on main — @opencode/core:test never ran, truncated run reads as ordinary failure #90

Activity

1. dezren39

   added

   kind/ciCI/CD pipelines, workflows, and build automationCI/CD pipelines, workflows, and build automation

   source/humanFiled directly by a humanFiled directly by a human

   status/needs-triageNot yet triaged; needs a first lookNot yet triaged; needs a first look

   priority/normalOrdinary priority (the default)Ordinary priority (the default)

   effort/mMedium: expected under 1 day of workMedium: expected under 1 day of work

   area/ciCI, workflows, and build infrastructureCI, workflows, and build infrastructure

   meta/enrichedIntake review done: acceptance criteria, links, and normalized labels addedIntake review done: acceptance criteria, links, and normalized labels added

   on Oct 26, 2026
2. dezren39 commented on Oct 26, 2026

   

   dezren39

   on Oct 26, 2026

   Author

   More actions

   Intake review

   🤖 Fleet monitor intake pass (protocol §5). The body already carries acceptance criteria, so the body was left untouched; this pass adds labels and this comment only.

   Related: run 39182188357 on main at b3e93cd (merge of PR #88 / #87 fix), detailed in the body; the two @opencode/sdk failures in that run were the already-tracked #62 assertions.

   Live check: still open and previously unlabelled. I did not find evidence in this pass that the turbo segfault has recurred or been diagnosed — equally, a single 2026-10-19 crash with no recurrence data may now be stale, which the next triage should test against recent unit (linux) runs before investing in a root-cause hunt.

   Notes / risks: the failure mode is dangerous precisely because it masquerades as ordinary test failures while suites (including @opencode/core:test) never run — a truncated run should fail loudly, not read as red tests.

   Suggested approach: first check recent main runs for any repeat SIGSEGV / exit-139 signature; if none, close-or-park with that evidence; if it recurs, treat making the truncation visible (job-level signal check) as the fixable part, with the turbo crash itself upstream.
3. dezren39 commented on Oct 28, 2026

   

   dezren39

   on Oct 28, 2026

   Author

   More actions

   Triage verdict: segfault NOT recurring on current main; the distinguishing measure is still missing

   Checked against live CI on current main (2026-10-28), not the issue text alone. The original sighting was run 39182188357 on b3e93cd0 (2026-10-19): turbo SIGSEGV, exit 139, @opencode/core:test never ran.

   No recurrence in sampled current-main runs (test workflow, unit (linux) logs grepped for SIGSEGV / Segmentation fault / exit code 139):

   • Run 40744745296 (b0633cc6): 0 crash matches; the @opencode/core:test group opened and executed — so acceptance criterion 1 (a later main head where core reports) is met.
   • Run 40741605903 (cb92f8fe): 0 crash matches, core:test present.
   • Run 40740624281 (af239842): 0 crash matches, core:test present.
   • Run 40737402873 (ad881d2b): unit (linux) success, 0 crash matches in the full log.

   Still open — criterion 3: at origin/main cd21eab1dec5, .github/workflows/test.yml's "Run unit tests" step is still a single bare GITHUB_ACTIONS=false bun turbo test --affected --concurrency=3 — no retry on exit 139, no distinct annotation, no turbo/bun version capture on crash. If turbo segfaults again, a truncated suite will again read as ordinary test failures.

   Route

   Scoped follow-up packet enqueued to repo-health-opencode-pilot as task 2693 (kind small_fix, dedupe repo-health:v1:fix:developing-today/opencode:issue:90): wrap the unit step so an orchestrator crash retries once and, on recurrence, is annotated distinctly with versions recorded; ordinary test failures unchanged. Criterion 4's close-with-evidence path applies once that measure lands — this issue stays open until then.

   Labels synced: status/needs-triage removed; queue/queued set for task 2693.
4. dezren39

   added

   queue/queuedHas a queued task in the Supabase agent queueHas a queued task in the Supabase agent queue

   and removed

   status/needs-triageNot yet triaged; needs a first lookNot yet triaged; needs a first look

   on Oct 28, 2026
5. dezren39 commented on Oct 28, 2026

   

   dezren39

   on Oct 28, 2026

   Author

   More actions

   🤖 Dispatcher binding · openchamber-dispatcher run 2026-10-28 15:30 CDT

   • Queue task: 2693 (repo-health-opencode-pilot, small_fix) ↔ issue CI: unit (linux) turbo run segfaulted mid-suite on main — @opencode/core:test never ran, truncated run reads as ordinary failure #90 (CI: unit (linux) turbo run segfaulted mid-suite on main — truncated run reads as ordinary failure)
   • Session: ses_e7bc22869ffe8u18n5anY1c60d (laptop OpenChamber, agent build) — the launch POST returned this id with promptDispatched: true, and the session listing independently binds it to the worktree/title below.
   • Pool: Workhorse (opencode-route/workhorse, resolving server-side to opencode-go / deepseek-v4.1-flash — OPEN). Fresh quota read this run ~15:32 CDT: opencode-go 5h 13.74% (reset 7:26 PM) / weekly 21.94% / monthly 73.45% — plan quota only, never PAYG.
   • Worktree / branch: ~/git/fleet/2693-90-turbo-crash-ci on the laptop, branch queue/2693-90-turbo-crash-ci (from origin/main cd21eab)
   • Verified engaged: session busy; its assistant messages (opencode-go deepseek-v4.1-flash) are reading the repo guidance and the current workflow file, then writing the wrapper script and updating the workflow step — the CI: unit (linux) turbo run segfaulted mid-suite on main — @opencode/core:test never ran, truncated run reads as ordinary failure #90 fix surface. The queue task itself remains queued in the fleet queue (no targeted claim/binding verb exists yet — fix(script): stats.ts fetches have no deadline and buffer unbounded .json() bodies — PostHog, npm downloads, and the GitHub releases pager #1337/fix(changelog): read upstream changelog through a 1 MiB capped reader #1272); the owning lane reconciles/harvests from here, and the babysitter shepherds the session.
6. dezren39 commented on Oct 28, 2026

   

   dezren39

   on Oct 28, 2026

   Last edited by dezren39

   Author

   More actions

   Task 2693 checkpoint: crash-retry wrapper implemented, PR open, CI reported

   Branch/commit: queue/2693-90-turbo-crash-ci @ 72ea2299 — PR #761 (opened, not merged): #761

   What changed

   • .github/workflows/test.yml, Run unit tests step only: both the --affected variant and the workflow_dispatch full-suite variant now invoke bash script/run-unit-tests.sh bun turbo test …. The step's env and flags are otherwise unchanged.
   • script/run-unit-tests.sh (new): runs the given command and inspects the outcome.
     • exit 139 or SIGSEGV anywhere in the attempt's output → ::warning title=Unit test orchestrator crashed::, then one retry.
     • retry crashes too → ::error title=Unit test orchestrator crashed twice:: stating that the run was truncated (so packages after the crash point, e.g. @opencode/core:test, may have no verdict) and recording turbo and bun versions, plus the same facts in the step summary. Fallback to the package.json pins if a crashed binary cannot answer --version.
     • retry passes → ::notice title=Unit tests passed on retry:: and a summary line, so a flake leaves a trace.
     • any other non-zero exit → ordinary test failure, passed through unrepeated, same exit code, no crash annotation.
     • Detection is deliberately the narrow one this issue names. A crash of a bun test process inside a finished turbo task already surfaces as a normal turbo task failure, which this measure does not reinterpret.
     • Turbo replays tasks completed before the crash, so the retry executes only the missing remainder of the suite.

   Simulation (laptop, stub commands, CI shell flags bash -eo pipefail, GITHUB_STEP_SUMMARY captured)

   • crash twice (stub exits 139 with Bun's Failed to run "turbo" due to signal SIGSEGV): warning → retry → distinct error annotation with turbo 2.10.2, bun 1.4.2 in both annotation and summary, exit 139.
   • crash once, then pass: warning → retry → notice, exit 0.
   • ordinary failure (exit 1, no signal text): one attempt, exit 1, no annotation, empty summary.
   • marker only (exit 1 with SIGSEGV printed): retried and annotated as a crash, exit 1 preserved.
   • success: one attempt, exit 0, no annotation.
   • bash -n and shellcheck clean; workflow YAML parses and the step resolves to the wrapper on both branches.

   CI on the PR (run 40765912824, base cd21eab1)

   • unit (linux) (job 123089886700) and unit (windows) (job 123089887027): the wrapper logged ==> unit tests, attempt 1/2: bun turbo test --affected --concurrency=3, made one attempt, and emitted no crash annotation — exactly the unchanged ordinary-failure path. Both jobs failed on the two permission-focus tests that unit: two permission-focus tests fail on linux + windows (pre-existing) #565 records as deterministic pre-existing failures on both platforms (@opencode/tui#test, exit 1).
   • Parity evidence: the same job on main at 88ecd395 — without the wrapper — failed with exactly those two tests (run 40762355777, Failed: @opencode/tui#test). The wrapper changes no ordinary verdict.
   • The PR run executed the full suite only because editing test.yml (a turbo global dependency) cache-invalidates every test task; that is expected for any workflow change and is why this PR surfaces unit: two permission-focus tests fail on linux + windows (pre-existing) #565 that affected-only main runs skip.
   • Other failing checks are unrelated and pre-existing too: e2e (linux/windows, app-2) regression specs, tracked by ci(app): e2e app-2 regression specs fail on subagent navigation/timeline visibility — reproduced on unrelated PR #607 (attempts 1+2) #631 as reproduced on unrelated PR fix(tui): truncators respect lengths 0-2 #607.
   • Passing: affected packages, build (linux/windows), typecheck, nix-eval, standards/compliance/duplicates, e2e app-1 and both component suites.

   State

   Implementation, simulation and CI reporting are done. PR #761 is open for review and must not be merged until checks are judged; its unit checks will stay red on #565 (tracked separately) regardless of this change. Nothing here requires a decision from #90's owners beyond merging the measure.

   Closure note

   Triage 2063 found no recurrence in 4 sampled recent main runs (runs 40744745296, 40741605903, 40740624281, 40737402873; @opencode/core:test present each time), and this measure now supplies criterion 3. I am not closing #90 myself; close it once PR #761 has merged and the measure is on main.
7. dezren39 commented on Oct 28, 2026

   

   dezren39

   on Oct 28, 2026

   Author

   More actions

   Closing per acceptance: (1) triage task 2063 found no recurrence of the turbo SIGSEGV in sampled current-main runs (runs 40744745296, 40741605903, 40740624281, 40737402873 — 0 segfault matches, @opencode/core:test executed); (2) PR #761 (merged a66dd47) adds the criterion-3 distinguishing measure: script/run-unit-tests.sh wraps both unit-step variants — a crash (exit 139 / SIGSEGV marker) retries once and, on recurrence, annotates distinctly (::error title=Unit test orchestrator crashed twice::) with turbo+bun versions in the annotation and step summary; ordinary failures are untouched.
8. dezren39

   closed this as completedon Oct 28, 2026
9. developing-today

   added

   status:done

   on Oct 28, 2026
10. dezren39

    removed

    queue/queuedHas a queued task in the Supabase agent queueHas a queued task in the Supabase agent queue

    on Oct 28, 2026