Skip to content

feat(aggregator): alert throat — sentinel intake, Supabase ledger, Web Push delivery + ack (#3676) - #3680

Merged
dezren39 merged 4 commits into
mainfrom
feat/alert-throat-3676
Nov 1, 2026
Merged

dezren39 merged 4 commits into
mainfrom
feat/alert-throat-3676

Conversation

@dezren39

@dezren39 dezren39 commented Nov 1, 2026 •

Copy link
Copy Markdown
Member

Implements #3676 (Drew approved 2026-11-01): every sentinel posts incidents to ONE intake; one engine owns dedupe, delivery, escalation, and ack; every transition lands in the Supabase alerts ledger. Delivery is the #1254 PWA Web Push design (slices a+b of #3315/#3316 with an incident class, ahead of #3317's dashboard classes).

Highlights:

  • PWA installability: manifest (start_url = attention view), generated icons incl. maskable + apple-touch, layout meta; PM_AGG_PUBLIC_ORIGIN defaults to the verified Tailscale Serve origin (443 → 127.0.0.1:4870).
  • Real /sw.js (push + notificationclick) with the design §6 fallback eviction carried forward (activate cache purge + page-side foreign-registration unregister via /assets/push-client.js); /api/sw-probe counts registration reports.
  • Subscription API over alerts.push_subscriptions (per-class map + quiet prefs round-trip; 404/410 gone-count pruning at 2). VAPID private key fetched by name from Bitwarden at runtime (memory only); sending via web-push.
  • POST /api/alerts intake (bearer = bws secret pm-aggregator.alerts-intake-token); dedupe rides the partial unique index; duplicate_suppressed event only when a delivery would otherwise repeat.
  • Engine tick (~60s): new open critical → immediate push (breakthrough class; quiet hours never hold it), delivered_at + notification_sends; unacked past ALERT_ACK_WINDOW_MIN (default 15) → exactly one escalation. Warnings: page + ledger only.
  • /alerts page with exactly-once ack + close; payloads deep-link to /alerts#incident-; scripts/post-alert.ts is the sender CLI; src/supabase.ts + src/secrets.ts reuse the collector's exact muse-dev2 access path.
  • Note: this branch also merges fix/feed-ts-missing-brace (PR fix(aggregator): restore closing brace dropped in the #2445 merge (main does not boot) #3679) — main does not boot without it; once fix(aggregator): restore closing brace dropped in the #2445 merge (main does not boot) #3679 lands this PR's diff is only the alert work.

Tests: full aggregator suite 870 pass / 0 fail on this branch (incl. 13 new alerts helper tests, the updated §6 worker test, a manifest test, and the curation/a11y/nav updates for the new page).


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

…n the #2445 merge

4d7f643 landed without the function's closing brace, leaving
src/feed.ts unparseable: the aggregator cannot boot from current
main at all (the running service survives only because it started
before the merge), and every test importing feed.ts errors. One
character, verified by the nav/curation/charges-feed-gate suites
going green again.
…b Push delivery + ack (#3676)

Implements #3676 (Drew approved 2026-11-01), building the #1254
design's slices (a) installability + (b) push plumbing with an
incident class, ahead of the dashboard classes in #3317:

- PWA: /manifest.webmanifest (start_url = the attention view),
  generated icons incl. maskable + apple-touch, install meta in the
  shared layout, PM_AGG_PUBLIC_ORIGIN defaulting to the Tailscale
  Serve HTTPS origin (verified: serve 443 -> 127.0.0.1:4870).
- /sw.js is the real worker (push + notificationclick) with the
  design §6 fallback eviction carried forward (activate cache purge
  + page-side foreign-registration unregister via the probe script);
  /api/sw-probe counts registration reports.
- Subscription API over Supabase alerts.push_subscriptions
  (per-class map + quiet prefs round-trip; 404/410 gone-count
  pruning at 2). VAPID private key fetched by name from Bitwarden
  at runtime, memory only; sending via the web-push package.
- POST /api/alerts intake (bearer token = the bws secret
  pm-aggregator.alerts-intake-token): dedupe rides the partial
  unique index; every transition writes alerts.incident_events.
- Engine tick (~60s): new open critical -> immediate Web Push to
  incident-class subscriptions, delivered_at + notification_sends
  rows; unacked critical past ALERT_ACK_WINDOW_MIN (default 15) ->
  exactly one escalation push. Warnings: page + ledger only.
- /alerts page: open incidents, exactly-once ack, close, ledger
  history, subscribe/preference controls; push payloads deep-link
  to /alerts#incident-<id>. scripts/post-alert.ts is the sender CLI.
- Supabase access reuses the collector's exact path (Management API
  SQL, bws supabase.access-token-muse-cli) in src/supabase.ts;
  secrets resolve by name in src/secrets.ts.

Tests: test/alerts.test.ts (13 pure-helper tests) green; boot test's
kill-switch assertion updated to the real worker per §6 + a manifest
test added. Full suite on this branch is additionally blocked by a
pre-existing main breakage (feed.ts brace dropped in #2445) — fixed
separately on fix/feed-ts-missing-brace.
…s page (#3676)

- curation-1497: record the Alerts page in the decision table at
  dropdown tier (post-#1497 addition, no primary slot without a new
  curation decision).
- nav #1135: /manifest.webmanifest joins NON_PAGE — PWA plumbing,
  not a navigable page.
- a11y #1215: the /alerts ledger table opens with a <caption> and
  scope="col" header cells like every other aggregator table.

Full suite: 870 pass / 0 fail.
@dezren39
dezren39 requested a review from a team as a code owner November 1, 2026 12:35
@coderabbitai

coderabbitai Bot commented Nov 1, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 58 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 9c3c76b3-0eb2-458d-8acd-bc7497ea9e03
Commits

Reviewing files that changed from the base of the PR and between 4ae8da5 and f7b849b.

Files ignored due to path filters: 5
  • pm-stack/aggregator/bun.lock is excluded by !**/*.lock
  • pm-stack/aggregator/src/assets/icons/apple-touch-icon.png is excluded by !**/*.png
  • pm-stack/aggregator/src/assets/icons/icon-192.png is excluded by !**/*.png
  • pm-stack/aggregator/src/assets/icons/icon-512.png is excluded by !**/*.png
  • pm-stack/aggregator/src/assets/icons/icon-maskable-512.png is excluded by !**/*.png
Files selected for processing: 19
  • pm-stack/aggregator/README.md
  • pm-stack/aggregator/package.json
  • pm-stack/aggregator/scripts/gen-icons.ts
  • pm-stack/aggregator/scripts/post-alert.ts
  • pm-stack/aggregator/src/alerts.ts
  • pm-stack/aggregator/src/assets/push-client.js
  • pm-stack/aggregator/src/config.ts
  • pm-stack/aggregator/src/feed.ts
  • pm-stack/aggregator/src/pages.ts
  • pm-stack/aggregator/src/push.ts
  • pm-stack/aggregator/src/render.ts
  • pm-stack/aggregator/src/secrets.ts
  • pm-stack/aggregator/src/server.ts
  • pm-stack/aggregator/src/supabase.ts
  • pm-stack/aggregator/src/sw-source.ts
  • pm-stack/aggregator/test/alerts.test.ts
  • pm-stack/aggregator/test/boot.test.ts
  • pm-stack/aggregator/test/curation-1497.test.ts
  • pm-stack/aggregator/test/nav.test.ts
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-01T12:40:07.332377Z f7b849b PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@dezren39
dezren39 merged commit 4e4c9f7 into main Nov 1, 2026
1 check passed
@dezren39
dezren39 deleted the feat/alert-throat-3676 branch November 1, 2026 12:35

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f7b849b7c0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +478 to +479
const summary = await deliverToAll(store, targets, incident, cfg.publicOrigin, false);
await store.markDelivered(incident.id, { ...summary });

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep failed deliveries eligible for retry

When every push attempt fails, such as during a push-provider outage, deliverToAll() returns a summary with zero successful sends, but the engine still sets delivered_at. This removes the incident from subsequent delivery ticks despite nobody receiving it. The escalation path likewise sets escalated_at after failed sends, permanently exhausting both notification opportunities if the outage spans the acknowledgement window. Only finalize a delivery or escalation after successful delivery, and retain failed attempts for retry.

Useful? React with 👍 / 👎.

Comment on lines +34 to +36
configurePromise = (async () => {
const priv = await resolveBwsSecret(VAPID_PRIVATE_KEY_SECRET);
if (!priv) return false;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Retry VAPID initialization after a failed secret lookup

If Bitwarden is temporarily unavailable on the first push attempt, this caches a promise resolving to false indefinitely. Subsequent calls reuse that promise instead of retrying secret resolution, even after the secret cache's five-minute expiry and Bitwarden recovery. The HTTP 401/403 reset cannot recover this state because sendPush() returns before contacting the provider. Clear unsuccessful initialization promises so a transient lookup failure does not disable all push delivery until the server restarts.

Useful? React with 👍 / 👎.

Comment on lines +51 to +54
if (!clientPromise) {
clientPromise = (async () => {
const token = await resolveBwsSecret("supabase.access-token-muse-cli");
if (!token) return null;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Allow the alerts store to recover from unavailable credentials

If the first management-token lookup fails temporarily, clientPromise permanently retains the resulting null, and getAlertsStore() additionally caches that unavailable result in storePromise. Consequently intake, acknowledgement, and the delivery engine remain unavailable after Bitwarden recovers; the periodic engine ticks never attempt initialization again. Retry unsuccessful initialization at both cache layers rather than requiring a process restart to restore the entire alert pipeline.

Useful? React with 👍 / 👎.

Comment on lines +5565 to +5566
<label><input type="checkbox" id="push-class-incident" checked> incident class</label>
<label>quiet <input type="time" id="push-quiet-start" value="23:00">–<input type="time" id="push-quiet-end" value="07:00"></label>

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Populate preference controls from the saved subscription

For an existing subscription, these controls always render the defaults rather than its saved preferences, and neither the inline script nor push-client.js subsequently loads those preferences. After disabling the incident class and reloading, changing only the quiet-hour start and clicking Save preferences submits incident: true and the default quiet-hour end, silently undoing the notification opt-out and other saved settings. Initialize the controls from the subscription matching this browser's endpoint before allowing preference updates.

Useful? React with 👍 / 👎.

const sub = await window.pmPush.current().catch(() => null);
status.textContent = sub ? "This browser is subscribed — pushes will land here." : "This browser is not subscribed yet.";
}
refreshStatus();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Check push support after the deferred client initializes

On every normal Alerts page load, this inline script calls refreshStatus() during HTML parsing, before the deferred /assets/push-client.js script emitted by layout() executes. Consequently window.pmPush is still undefined and the page reports that Web Push is unsupported even in supported, already-subscribed browsers. Nothing refreshes the status after the client initializes. Run this initialization after the deferred script, such as on DOMContentLoaded, so the displayed delivery status reflects the browser's actual subscription.

Useful? React with 👍 / 👎.

Comment on lines +346 to +350
);
if (rows[0]) {
await this.addEvent(id, "acked", {});
await this.db.query(
`update alerts.notification_sends set acted_on_at = now() where incident_id = ${sqlStr(id)} and acted_on_at is null`,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Commit acknowledgement state and ledger writes atomically

If the acknowledgement event insertion fails after the preceding status update succeeds, the request fails but the incident remains acked. A retry cannot repair the missing event or stamp notification_sends.acted_on_at: the update's status = 'open' predicate no longer matches, so both follow-up writes are skipped permanently. Execute the status transition, event insertion, and send updates in one database transaction so a transient query failure cannot leave an acknowledged incident without its durable acknowledgement record.

Useful? React with 👍 / 👎.

objectRef: incidentObjectRef(incident.id),
url: incidentUrl(origin, incident.id),
title: escalation ? `Escalation: ${incident.title}` : incident.title,
body: escalation ? `Still unacknowledged: ${base} (source: ${incident.source})` : `${base} (source: ${incident.source})`,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Bound notification content to the Web Push payload budget

For incidents containing long diagnostic output, intake accepts the complete body and this function embeds it unbounded into the push payload. A valid 4,000-byte diagnostic body already makes the plaintext JSON exceed the 3,993-byte budget supported by services enforcing the standard 4,096-byte encrypted-payload limit, so those providers reject the notification regardless of retries. RFC 8291 §4 specifies these limits. Apply a UTF-8 byte budget to the complete serialized notification, truncating its display text while retaining the full diagnostics in the ledger and linked incident page.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant