Repository navigation
quota: /quota roster flickers — two fleet-feed producers write divergent quota sections (disabled-filter + code-rev skew) #3852
Description
Bug: /quota roster flickers — two fleet-feed producers write divergent quota sections to the same file
Found in the 2026-11-01 PM-stack page-by-page review.
~/.local/state/pm-aggregator/fleet-feed.json has TWO producers running the same collector (scripts/collect-fleet-feed.ts) on a ~5 min cadence:
- The laptop's
fleet-feed-collector.timer— reads the LAPTOP's deployed opencode config for the fix(usage-hub): disabled providers must not appear in quota — audit + regression test #1114 disabled_providers filter and runs current code. - A sandbox-side push (
scripts/push-fleet-feed.sh, "systemd, sandbox" per its header) — runs the collector against the sandbox's code checkout and the sandbox's config, then overwrites the laptop file.
Their quota sections diverge, so /quota's roster changes depending on which producer wrote last. Measured live 2026-11-01:
- 16:58Z file: 11 pools, no
openai.alternate(sandbox-shaped: unfiltered disabled-door pools present). - 17:01Z file (laptop collector, post-fix(quota): surface the alternate ChatGPT account as its own pool on /quota #3837): 6 pools —
openai.alternatepresent; meta:high-usage, anthropic:claude-pro, commandcode:go, antigravity:gemini, antigravity:claude-gpt, deepseek:api filtered per fix(usage-hub): disabled providers must not appear in quota — audit + regression test #1114 (the laptop config disables their doors: meta, claude-code, commandcode, deepseek, antigravity-ls+agy+google). - 17:03Z file (sandbox push): back to 11 pools, alternate gone again.
The laptop collector's journal shows the #1114 filtering firing all day (612 "filtered —" lines on 2026-11-01, 0 unreadable-config fallbacks), while the sandbox producer evidently assembles with a different (emptier) disabled set and, until its checkout syncs, older code.
Why it matters
- Pools appear and disappear from /quota every few minutes — this is the direct cause of "where's alternate openai and so on": whether a pool is visible depends on refresh timing.
- The fix(usage-hub): disabled providers must not appear in quota — audit + regression test #1114 filtered pools are "noted" only in the laptop collector's journal; the page/API carries no trace of them, so a reader cannot distinguish "pool filtered because its door is disabled" from "pool missing".
Suggested direction (needs an owner decision)
- Give the quota section a single producer (the laptop — the pools describe laptop-served providers and the laptop's deployed config is the fix(usage-hub): disabled providers must not appear in quota — audit + regression test #1114 source of truth), and have the sandbox push preserve the existing quota section the way the AA section is already reused/carried; OR make both producers read the same config + code rev (fragile).
- Surface the fix(usage-hub): disabled providers must not appear in quota — audit + regression test #1114 filtered pools on /quota itself (a "filtered — door disabled" note listing them), not only in the collector journal.
Acceptance criteria
- The fleet-feed quota section has a single effective producer: either the laptop collector alone writes it, or the sandbox push (
scripts/push-fleet-feed.sh) preserves the existing quota section the way it already carries the AA section. - Two consecutive producer cycles (~10 minutes apart) yield an identical /quota pool roster — no pool appears or disappears depending on which producer wrote last, and the roster matches the laptop's fix(usage-hub): disabled providers must not appear in quota — audit + regression test #1114 disabled_providers filtering.
- Pools filtered by the fix(usage-hub): disabled providers must not appear in quota — audit + regression test #1114 disabled-door rule are surfaced on /quota itself (e.g. a "filtered — door disabled" note naming them), not only in the collector journal, so "filtered" is distinguishable from "missing" on the page.
Metadata
Metadata
Assignees
Labels
Package/app: aggregator (pm-stack in this repo)pm-stack (Pad/Grist) in developing-today/codeMedium: expected under 1 day of workSomething isn't working (prefixed form of the classic bug label)Intake review done: acceptance criteria, links, and normalized labels addedImportant and time-sensitiveFiled by the new-issue-creator jobQueue packet kind: verify, reproduce, and route this item
Activity
dezren39 commentedon Nov 1, 2026
Update from the 2026-11-01 page-by-page review: the flicker half of this issue is RESOLVED.
The divergent second writer was the sandbox host's system-level
fleet-feed-collector.timerrunningpush-fleet-feed.shfrom a stale~/git/pm-feedclone (HEAD 93a8d8e, 2026-10-29) — the phantom writer already tracked as #3362. It has been retired: PR #3865 (merge d8ead56) deletes the sandbox-side collector units + installer from this repo and marks push-fleet-feed.sh manual-only; the sandbox timer was stopped/disabled and the~/tools/init.shrestore block replaced with a retirement note so a sandbox recycle cannot resurrect it. (Same root cause as the /aa "Supabase 403" — the stale clone's default project ref is the retired muse-dev project while the management token is muse-dev2-scoped; details on #3362 / #3294.)Verified 2026-11-01 ~17:15–17:27Z: the laptop collector is now the sole writer across multiple ticks and two dead sandbox push windows; /api/quota.json is stable at 6 pools including
openai.alternate(the new pool from #3837), and /api/aa.json isstatus: okwith live data.What REMAINS open on this issue is only the second ask: the #1114-filtered pools (currently meta:high-usage, anthropic:claude-pro, commandcode:go, antigravity:gemini, antigravity:claude-gpt, deepseek:api — filtered because the deployed config disables their door providers) are noted only in the collector's journal. /quota itself carries no trace of them, so a reader cannot distinguish "filtered — door disabled" from "missing". Surfacing them on the page (e.g. a "filtered — door disabled" line) is a small change but a visible-behavior decision; leaving this issue open for that call.
dezren39 commentedon Nov 2, 2026
Intake review
openai.alternate— the pool whose disappearance is the visible symptom), PR fix(quota-gatherer): bound provider concurrency + retain last-known snapshot on error (#1268) #3996 (merged 2026-11-01, quota-gatherer concurrency/snapshot retention — adjacent but a different component; it does not touch the dual fleet-feed producers), and the fix(usage-hub): disabled providers must not appear in quota — audit + regression test #1114 disabled_providers filter that is the divergence source. No merged PR resolves the dual-producer overwrite.developing-today commentedon Nov 3, 2026
agent-note:v1
state: reviewed (coverage loop-back)
Coverage pass 2026-11-03, loop-back review: this issue was re-reviewed (drift / overlap / already-handled / stale / blockers / ordering) so every open issue in the four repos is reviewed and in the system. Verdict: still live; in the system as task 7416 (triage, queued, priority 50) in repo-health-code-pilot. Labels audited and synced. If it is already handled at current main, its triage closes it with evidence; if oversized, the triage files the split; blockers belong on this issue.
Found via: coverage loop-back 2026-11-03
dezren39 commentedon Nov 4, 2026
agent-note:v1
event: triaged
task: 7416 (repo-health-code-pilot, repo-worker-code; OpenChamber scheduled lane 1a180ae5-384f-4e71-8d77-08f7b1652c75)
state: Verified at current origin/main 12fcad9 that both issue asks are superseded/resolved. The dual producer was retired by merged PR #3865 (merge d8ead56, 2026-11-01). The remaining filtered-pools visibility ask became obsolete under Drew’s explicit 2026-11-03 ruling implemented by merged PR #5080 (merge f508cfb): the dashboard passes through all pools from the quota read and does not consult disabled_providers; tests pin all-pool pass-through. PR #5080 is an ancestor of current main. PR #5080 reports aggregator tests 1390 pass / 0 fail. #3362 remains open for its separate >=1h all-writes observation criterion, but that does not block #3852.
next: Closing #3852 as superseded by merged #3865 and #5080; no separate follow-up needed.
links: #3852 · #3362 · #3865 · #5080
dezren39 commentedon Nov 4, 2026
Superseded/resolved: #3865 removed the divergent sandbox fleet-feed producer (merged 2026-11-01), resolving the roster flicker. The remaining request to display disabled-door filtered pools is obsolete after Drew’s 2026-11-03 direction and #5080: the dashboard now passes through every pool in the quota read and no longer filters by disabled_providers. Verified #5080 merge f508cfb is an ancestor of current main 12fcad9; its tests report all-pool pass-through. See the preceding agent-note:v1 triage record. #3362 remains open only for its separate direct-observation acceptance item.
dezren39 commentedon Nov 4, 2026
agent-note:v1
event: done
task: 7416 (repo-health-code-pilot, repo-worker-code; OpenChamber scheduled lane 1a180ae5-384f-4e71-8d77-08f7b1652c75)
state: Closed as superseded/resolved after fresh verification: #3865 retires the second writer; #5080 removes disabled-provider filtering and satisfies the obsolete visibility ask. #5080 is an ancestor of current main 12fcad9. Queue task 7416 completed as superseded. Lifecycle labels cleared.
next: None.
links: #3852 · #3362 · #3865 · #5080