Skip to content

[P2] A pr-fix item marked FIXED on workload exit alone strands the PR forever, with no way back #940

Closed

Description

@joryirving

Problem

A PR-fix item is marked FIXED when the foreman fix Workload exits successfully. Nothing checks that the pull request actually advanced. When a Workload succeeds without pushing a fix, the item leaves the queue as done, the PR keeps its CHANGES_REQUESTED, and nothing ever retries it — a FIXED item is not re-dispatched, and there is no API path back.

Worse, the condition stays visible the whole time: pr-followup re-detects it every 15 minutes and writes a fresh enqueue history row, so the system can see the PR is still blocked and does nothing about it.

Worked example

misospace/miso-gallery#467 (issue #447), stuck 34 hours:

2026-09-04 11:11  enqueue  PR review: CHANGES_REQUESTED
2026-09-04 11:30  mark FIXED  "foreman fix Workload prfix-misospace-miso-gallery-467 succeeded"
2026-09-04 11:41  enqueue  PR review: CHANGES_REQUESTED     <- PR never changed
2026-09-04 11:56  enqueue
...
2026-09-05 20:45  enqueue      (272 enqueue rows, 1 mark row)

The PR still carries its single original commit efc36e3d and the CHANGES_REQUESTED review from 2026-09-04T11:04:17Z. No fix commit was ever pushed. All CI checks pass, so nothing else flags it either.

The underlying review was correct and substantive — the coder added tests only and left the app.py symlink guard unimplemented — so this is a case where a second pass was genuinely needed and never happened.

Why the check cannot currently be made

PrFixQueueItem.headSha is null on this item. Dispatch records no head SHA when it enqueues, so even a FIXED transition that wanted to verify "did the PR move" has nothing to compare against.

Second defect: no recovery path

A wrongly-FIXED item cannot be returned to the queue. requeue_pr_fix refuses:

Error: Cannot requeue: item is FIXED, not BLOCKED

The only route is mark_pr_fix status=blocked followed by requeue_pr_fix — two calls, where the first is a lie about the item's state, and the intermediate blocked also flips the lane to NEEDS_HUMAN before requeue restores it. That is what I had to do to unstick #467.

Suggested shape

  1. Record headSha at enqueue so a comparison is possible at all.
  2. Verify before marking FIXED: the PR head must differ from the recorded head. A Workload that exits successfully without moving the head is not a fix — treat it as BLOCKED (or leave it QUEUED against the attempt budget) rather than done.
  3. Reopen a stale FIXED: when pr-followup re-detects the same evidence key on an item already marked FIXED whose PR head has not moved, put it back in the queue instead of writing another enqueue row. This is the safety net for however the first two are implemented.
  4. Optionally let requeue_pr_fix accept a FIXED item directly, so recovery is one honest call rather than two.

Point 3 alone would have caught this: the evidence was re-detected 271 times after the item was closed out.

Files

  • src/app/api/pr-fix-queue/mark/route.ts — the FIXED transition; no head comparison
  • src/app/api/pr-followup/sync/route.ts — re-detects and re-records enqueue for items already FIXED
  • src/lib/pr-followup-ingestion.ts — computeEvidenceKey and the enqueue gating
  • prisma/schema.prisma — PrFixQueueItem.headSha, currently unpopulated

Scope note

Nothing here is Foreman-specific. "A fix workload ran and the PR did not move" is a plain statement about a PR and any external fixer, and the fix should stay that way.

Activity

  1. added
    auditAudit, review, or investigation work.
    type/bugSomething isn't working
    status/readyReady for Dispatch worker pickup.
    on Sep 5, 2026
  2. added a commit that references this issue on Sep 5, 2026
    6d70e87
  3. added a commit that references this issue on Sep 6, 2026
    a8edb41
  4. 11 remaining items

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions