Repository navigation
fleet binary: fallback verb exists in source + docs but is missing from the installed 0.1.0 build (sandbox and laptop) #6728
Description
Problem
doc/fleet/queue-fallback-mode.md (Drew's order 2026-11-06, code#6619) declares fallback mode ACTIVE with tooling fleet queue fallback (pkgs/fleet/crates/fleet-queue/src/fallback.rs), and the queue worker instruction §0a routes every backend-down shift through it. But the installed fleet 0.1.0 binary does not contain the verb:
- Sandbox:
fleet queue fallback scan --repo developing-today/code→repo-queue: unknown command "fallback";fleet queue --helplists only validate, list, enqueue, claim, lane, bind, heartbeat, checkpoint, park, block, cancel, fail, complete, health, signal. - Laptop (
/.local/bin/fleet, also 0.1.0): identical command list — while fallback.rs IS present in the same clone's source tree (/git/code/pkgs/fleet/crates/fleet-queue/src/fallback.rs on main).
So the fallback system was merged as source + docs but never built/deployed into the binary workers actually run. During the 2026-11-06/07 PostgREST outage (PGRST002, health exit 4 for hours), Lane G (2026-11-06 20:23 CST shift) had to hand-post the doc's claim/heartbeat/release markers via gh to work issue #6716 in fallback mode — workable because the protocol is fully specified in the doc, but every lane must improvise it, and the tool's race re-reads / contested-claim checks (exit 5 semantics) do not exist in the hand-rolled version.
Evidence (verified 2026-11-07 ~03:11-03:15Z)
fleet queue --helpoutput above on both sandbox and laptop installs.- fallback.rs present in laptop primary clone source on main.
- Lane G fallback work on repo-universe(generate): wrong-shape enumeration entries crash the whole generator — a non-object entry or a dict without full_name aborts regeneration with a bare TypeError/KeyError #6716 used manual markers (claim 03:13:24Z, release 03:23:02Z) and landed as PR fix(repo-universe): name wrong-shape enumeration entries instead of crashing (#6716) #6723 (merged).
Acceptance
- A released fleet build whose
fleet queue --helpincludes fallback, deployed to BOTH the sandbox PATH and the laptop ~/.local/bin/fleet. -
fleet queue fallback scan --repo developing-today/codereturns JSON counts/items against live GitHub state. - The deploy path (which build artifact feeds both installs) is documented where the next binary change will find it, so a merged verb cannot silently stay undeployed again.
Activity
developing-today commentedon Nov 7, 2026
CODE-2081
dezren39 commentedon Nov 7, 2026
Picked up by repo-worker-code-z (lane code-z, repo-queue-worker-shift-z) in FALLBACK MODE (backend down: fleet queue health HTTP 503 PGRST002 — and the fallback verb itself missing from this binary, which is the subject of this issue). Triage of issue #6728: verify installed-binary verb lists on sandbox + laptop, fallback.rs presence/wiring in source on current main, and locate the binary deploy path; route the build/deploy fix.
🤖 Agent note · picked up
dezren39 commentedon Nov 7, 2026
🤖 Agent note · done (triage verdict: CONFIRMED for the sandbox half; the laptop half RESOLVED since filing)
Verdict: PARTIALLY RESOLVED — sandbox install still stale, laptop install fixed after filing
Fresh verification this shift (2026-11-07 ~03:2xZ):
fleet→~/tools/bin/fleet→~/tools/fleet(Nov 4, root-owned, read-only).fleet version: 0.1.0, source rev18f0aa1a68d83d3df127cca8523f578f4adb33a3.fleet queue --helplists validate/list/enqueue/claim/lane/bind/heartbeat/checkpoint/park/block/cancel/fail/complete/health/signal — no fallback;fleet queue fallback scan→repo-queue: unknown command "fallback"(reproduced at this shift's start and again now).~/.local/bin/fleetrebuilt 2026-11-06 21:21:50 -0600, source revff160b996476, and its queue command list now INCLUDES fallback. Verified live:fleet queue fallback scan --repo developing-today/codeon the laptop returns JSON items against current GitHub state. So acceptance items 1 (laptop half) and 2 are already satisfied — by a deploy that landed ~03:21Z, after the evidence in the issue body was gathered.pkgs/fleet/crates/fleet-queue/src/lib.rs:31 pub mod fallback;,client.rsusage string includesfallbackand dispatch maps"fallback" => crate::fallback::run(...)— the verb is fully wired in source; this was purely a build/deploy gap.pkgs/fleet/README.md"Build and smoke" —nix build .#fleet .#fleet-serve .#fleet-connect .#fleet-connect-static+smoke.sh. The laptop install is a plain binary at~/.local/bin/fleet. The sandbox install is NOT lane-writable:~/tools/fleetand the~/tools/bin/{fleet,repo-queue}symlinks are root-owned provisioning artifacts, so the sandbox redeploy belongs to whoever provisions~/tools, or to a documented re-point of those symlinks at a fresh artifact.Routing
Returned to
queue/queuedastask/build-deploy+status/triaged. Remaining work, in order: (1) redeploy a current-rev fleet build to the sandbox~/tools/fleet(provisioning-owned — a lane cannot write it), verifyingfleet queue --helplists fallback and a sandboxfallback scanreturns JSON; (2) document the two install points + build command where the next binary change will find it (pkgs/fleet/README deploy section or doc/fleet/queue-fallback-mode.md), per acceptance item 3. Until (1) lands, fallback shifts keep hand-posting markers via gh — workable, as this shift's completions show, but without the tool's race/contest checks.(Comment repaired in place by its author: the first posting's backtick spans were mangled by a shell heredoc error; the substance is unchanged.)
dezren39 commentedon Nov 7, 2026
Released free by repo-worker-code-z (lane code-z, FALLBACK MODE): triage of issue #6728 complete — verdict on the issue (sandbox half CONFIRMED stale at rev 18f0aa1; laptop half RESOLVED since filing, new build rev ff160b9 with working fallback scan). Fix routed on the same issue: task/build-deploy + status/triaged + queue/queued (sandbox redeploy is provisioning-owned; deploy-path documentation owed).
56 remaining items
dezren39 commentedon Nov 7, 2026
🤖 Agent note · done (lane scope) — one acceptance item remains, owned by provisioning
pkgs/fleet/Cargo.lockregenerated (+67/−1) — it predated atlas-core'ssha2 0.10dependency, sonix build .#fleetfailed in the vendor step and NO current build could be produced at all; plus a README Deploy-section note naming this lockfile-drift failure and its regeneration step. (The deploy-path documentation itself landed earlier in bc39d0b.)nix build .#fleetfrom the fixed tree → artifact renamed over~/.local/bin/fleet(rename, since the running broker holds the old inode; pre-deploy copy at~/.local/bin/fleet.bak-pre6728).fleet queue --helplistsfallback;fleet queue fallback scan --repo developing-today/codereturned live JSON, exit 0, at 2026-11-07T04:30:57Z.~/tools/fleetreports source revff160b996476; current main is17eb1cbada+ the lock fix above (merge 81bdda4). Per the README Deploy section this file is root-owned provisioning and lanes do not redeploy it — both revs are recorded here for the provisioning owner, exactly as that section prescribes. This is the one open acceptance item; the issue stays open and re-queued so it keeps surfacing until the sandbox binary moves.dezren39 commentedon Nov 7, 2026
Released by
repo-worker-code-i(outcome: free). lane scope complete: Cargo.lock build break fixed + merged (PR #6774), laptop install deployed and verified live (fallback scan exit 0); sandbox install still stale at rev ff160b9 vs main 17eb1cb+fix — root-owned provisioning per README deploy section, revs recorded on the issue; re-queued so it keeps surfacing until provisioning redeploysdezren39 commentedon Nov 7, 2026
Picked up by
repo-worker-code-a— working this now. (Fleet queue fallback mode: the queue backend is down, so this issue's comments carry the claim. The claim stays live whilerepo-worker-code-aposts progress markers; after 30 minutes of silence any other worker may pick this up.)dezren39 commentedon Nov 7, 2026
Progress from
repo-worker-code-a: claimed: verifying fallback verb presence in installed fleet builddezren39 commentedon Nov 7, 2026
🤖 Agent note · already done — acceptance re-verified live by this lane
fleet queue --helplistsfallback, and laptop~/.local/bin/fleet queue --helpalso listsfallback(fresh laptop read this shift); (2)fleet queue fallback scanreturned live JSON, and this lane has run scan/claim/heartbeat/release through the real verb for 20 tasks this shift — the verb is deployed and working, not just present; (3) deploy path documented (PR docs(fleet): document the two live installs + deploy verification (#6728) #6732; README deploy section). Trailing nuance from lane code-i (04:39Z): the sandbox install's rev trails newest main (root-owned provisioning seat) — a rev-currency point, not an acceptance failure; the verb this issue exists to ship is live on both seats.queue/claimed+work/assigned.dezren39 commentedon Nov 7, 2026
Released by
repo-worker-code-a(outcome: done). Already done (re-verified live): #6728 CLOSED 2026-11-07T04:05:41Z; sandbox and laptop fleet builds both list the fallback verb (fresh reads this shift), fallback scan returns live JSON, and this lane executed 20 fallback claims/releases through it this shift. Deploy path documented (PR #6732). Stale queue/claimed + work/assigned labels removed.