Skip to content

fix(aggregator): retire the sandbox fleet-feed collector units (#3362) - #3865

Merged
dezren39 merged 1 commit into
mainfrom
fix/3362-retire-sandbox-fleet-feed-collector
Nov 1, 2026
Merged

dezren39 merged 1 commit into
mainfrom
fix/3362-retire-sandbox-fleet-feed-collector

Conversation

@dezren39

@dezren39 dezren39 commented Nov 1, 2026 •

Copy link
Copy Markdown
Member

Root cause (code#3362 phantom writer, identified)

The writer overwriting the laptop fleet feed with an AA-absent (Supabase HTTP 403) section every ~5 min is the **sandbox host's own ** (systemd, system level), running from the stale clone (HEAD 2026-10-29 , pre-keep-last-good #3349), restored on every sandbox boot by .

Its AA 403, reproduced directly from the sandbox with its own bws token (secret name , name only):

  • token + retired muse-dev ref (the stale code's default; its installed unit predates the 2026-10-31 muse-dev2 pin 5885986) -> HTTP 403 {"message":"Your account does not have the necessary privileges to access this endpoint."}
  • same token + muse-dev2 ref -> HTTP 201, current batch

Not RLS/grants: the token is muse-dev2-scoped and the stale copy queries the retired project. The sandbox journal shows it plainly: -> , landing in via the symlink, racing the laptop collector that has owned the feed since the 2026-10-31 move (#3363/#3368). Its other sections are also degraded (sandbox OpenCode server unreachable -> jules-only sessions; unfiltered pools).

Fix

The collector lives on the laptop now (pm-stack NixOS module), so the sandbox-side units are deleted, not updated:

  • delete and
  • : header marked RETIRED as a scheduled producer / manual-only (script kept: it carries the charges confirm-after-delivery contract, fix(aggregator): confirm surfaced charges after delivery #3167, and covers it)
  • README + + + : stale "sandbox collector / pushed feed" descriptions corrected to the laptop collector

Activation (done alongside, outside the repo): sandbox stopped + disabled, installed units removed from , and the restore block in the sandbox replaced with a retirement note so a sandbox recycle cannot resurrect it.

Tests

bun test v1.4.2 (744846f84) in : 870 pass / 2 fail — identical on pristine origin/main (pre-existing failure). No new failures.

Closes the AA-403 half of #3294; identification + evidence recorded on #3362.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

The phantom writer blanking /aa is identified: the sandbox host still
runs fleet-feed-collector.timer (systemd, system level), installed from
the stale ~/git/pm-feed clone (HEAD 2026-10-29, pre-#3349) and restored
on every sandbox boot by ~/tools/init.sh. Its collector defaults
SUPABASE_PROJECT_REF to the retired muse-dev project
(nltgxorqwxhgfzseiirt) while the bws management token
(supabase.access-token-muse-cli) is muse-dev2-scoped, so its AA query
fails HTTP 403 (verified directly: same token + retired ref = 403
"does not have the necessary privileges"; same token + muse-dev2 ref
keihfchslgjetkmnmzra = 201 with the current ainventory batch). Every
~5 min it scp-pushed its AA-absent, session-poor feed over the laptop
collector's good feed (~/pm-stack/aggregator symlinks into the
aggregator state dir), racing the laptop collector that has owned the
feed since the 2026-10-31 move (#3363/#3368).

The collector lives on the laptop now (pm-stack NixOS module), so the
sandbox-side units are deleted, not updated: remove
systemd/fleet-feed-collector.{service,timer} and
scripts/install-collector-units.sh, mark push-fleet-feed.sh
manual-only, and correct the stale "sandbox collector / pushed feed"
descriptions in README, src/feed.ts, src/topology.ts and
src/watchdog.ts. The charges confirm-after-delivery contract stays:
push-fleet-feed.sh + confirm-charges-surfaced.ts are untouched.

Tests: bun test in pm-stack/aggregator — 870 pass, 2 fail, identical
on pristine origin/main (pre-existing slice-3 routes failure).
@dezren39
dezren39 requested a review from a team as a code owner November 1, 2026 17:11
@coderabbitai

coderabbitai Bot commented Nov 1, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 46 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 13aaf498-3c10-46c8-9d6a-66a48ba13567
Commits

Reviewing files that changed from the base of the PR and between f1ddfb3 and d6552b8.

Files selected for processing: 8
  • pm-stack/aggregator/README.md
  • pm-stack/aggregator/scripts/install-collector-units.sh
  • pm-stack/aggregator/scripts/push-fleet-feed.sh
  • pm-stack/aggregator/src/feed.ts
  • pm-stack/aggregator/src/topology.ts
  • pm-stack/aggregator/src/watchdog.ts
  • pm-stack/aggregator/systemd/fleet-feed-collector.service
  • pm-stack/aggregator/systemd/fleet-feed-collector.timer
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-01T17:12:38.706082Z d6552b8 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant