Skip to content

worktree-reaper: fail closed when /proc cannot be scanned before removal #5740

Open

Description

@dezren39

Problem

The worktree-reaper can treat a process-scan failure as proof that a worktree is unoccupied, then pass its final removal gate. In infra/worktree-reaper/bin/worktree_reaper.py, the /proc scan helper around lines 377–394 returns an empty process list when os.listdir("/proc") fails; tree_occupied around lines 397–403 interprets that empty list as unoccupied. The fresh pre-removal check around lines 479–487 therefore fails open on an inspection error.

This is distinct from the recent run-log collision fix in #5716/#5718. The reaper is destructive, so “could not determine” must not mean “safe to delete.”

Required behavior

Represent process-scan failure distinctly from a successful scan with zero matching processes. If occupancy cannot be established, retain the candidate and emit a clear warning/error; do not call git worktree remove, remove the branch, or delete the path.

Acceptance criteria

  • Inject an os.listdir("/proc") failure and assert the candidate is treated as occupied/unknown and retained.
  • The failure-path test asserts git worktree remove and branch deletion are not called.
  • Preserve the existing behavior for a successful empty scan and for a confirmed matching process.
  • The test is discovered by the documented CI/test command for infra/worktree-reaper/tests/test_worktree_reaper.py.

Activity

  1. added
    kind/bugSomething isn't working (prefixed form of the classic bug label)
    priority/highImportant and time-sensitive
    area/infraInfrastructure, deployment, and operations
    on Nov 5, 2026
  2. developing-today commented on Nov 5, 2026

    @developing-today
    Contributor
  3. added
    queue/queuedHas a queued task in the Supabase agent queue
    task/triageQueue packet kind: verify, reproduce, and route this item
    queue/claimedQueue worker lifecycle state
    work/assignedQueue worker lifecycle state
    and removed
    queue/queuedHas a queued task in the Supabase agent queue
    on Nov 5, 2026
  4. dezren39 commented on Nov 5, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · handed off

  5. added
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    status/parkedDeliberately paused with a recorded resume condition
    on Nov 5, 2026
  6. 196 remaining items

  7. dezren39 commented on Nov 7, 2026

    @dezren39
    MemberAuthor

    Progress from repo-worker-code-q: verified + branch_created: bug CONFIRMED fresh on origin/main bcc0901 (proc_cwds at infra/worktree-reaper/bin/worktree_reaper.py:377-384 still returns the empty list on OSError; both consumers fail open, exactly as lane code-k's triage found on 90a5463). No competing fix PR (only PR #6357 for sibling issue #5946, workflow file only). Laptop worktree ~/git/fleet/fallback-5740 on branch queue/fallback-5740-proc-fail-closed created from origin/main. (Laptop primary ~/git/code carries unrelated opencode-claimant dirt from another lane — left untouched; worktree is from fresh origin/main.) Next: implement fail-closed scan handling + failure-injection tests.

  8. dezren39 commented on Nov 7, 2026

    @dezren39
    MemberAuthor

    Progress from repo-worker-code-q: implementation_complete + targeted_tests_passed: proc_cwds now returns None on a /proc listing failure (unknown ≠ empty); tree_occupied fails closed on None; evaluate_tree keeps the candidate with reason occupied-unknown (gate error proc-scan-failed); reap_tree re-scans fresh and keeps with occupied-unknown-at-removal before any removal; main emits a stderr warning + summary error naming the fail-closed retention. Tests: 4 new in infra/worktree-reaper/tests/test_worktree_reaper.py (unit: scan failure is None / unknown counts as occupied while empty-scan and confirmed-match behavior is preserved; gate: failure injected at evaluation and at the removal re-scan, candidate tree + branch asserted retained on disk). Documented command python3 tests/test_worktree_reaper.py on the laptop: 12 passed, 0 failed (8 pre-existing + 4 new). The new tests are discovered by that documented command; wiring this suite into a workflow is sibling issue #5946 / PR #6357. Next: commit, push, open PR.

  9. dezren39 commented on Nov 7, 2026

    @dezren39
    MemberAuthor

    Released done by repo-worker-code-z (lane code-z, FALLBACK MODE): PR #6731 merged — merge commit 5df7c86 — and issue #5740 closed by the merge. proc_cwds raises ProcScanError on scan failure (distinct from a successful empty scan); reap_tree retains the candidate (occupied-unknown-at-removal, warning, zero git calls) and main aborts exit 1 when the initial scan fails. Tests on laptop: python3 tests/test_worktree_reaper.py — 12/12 (4 new fail-closed tests incl. run_git-never-called assertion). Codex review on head 0d1f3bf: completed, no findings. scripts-test checks QUEUED in the fleet-wide runner backlog — merged under standing merge authority with local test verification.

    🤖 Agent note · done

  10. dezren39 commented on Nov 7, 2026

    @dezren39
    MemberAuthor

    Released by repo-worker-code-q (outcome: free — superseded, not unworked). Issue #5740 was fixed and closed by lane code-z's PR #6731 (merged 2026-11-07T03:34:00Z, merge commit 5df7c86); lane code-z already posted the outcome=done release of record at 03:34:16Z. This release closes out lane code-q's parallel claim (claimed 03:29:24Z, 24s after code-z's — a fallback-mode claim race: the deployed fleet binary has no fallback claim verb to refuse a second live claim).

    🤖 Agent note · superseded (duplicate work, closed per protocol)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

app/worktree-reaperinfra/worktree-reaperarea/infraInfrastructure, deployment, and operationskind/bugSomething isn't working (prefixed form of the classic bug label)priority/highImportant and time-sensitivestatus:in-reviewtask/small-fixQueue packet kind: bounded fix with reproduction and targeted tests

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions