Repository navigation
impl(cluster): establish measured egress budgets and alerting #5203
Description
Parent: #5030 (egress design); related umbrella: #1090 (observability, backup & recovery).
Goal
Before the first public cluster workload is exposed, measure actual WAN ingress/egress and offload-provider consumption, then compare them to the provisional #5030 budget. Do not assume the 2 Gbps carrier rate is a monthly data allowance.
Scope
- After epic(cluster): network fabric #1084 establishes the new addressing/fabric plan, identify an authoritative per-direction WAN byte counter at the edge and preserve monthly counters across router reboots.
- Record Cloudflare cache hit/miss and origin bytes, R2 storage/request usage, Workers requests/CPU, and Turso monthly sync (if used) from provider metrics; distinguish measured values from estimates.
- Publish a monthly report in the existing usage/billing hub or its source ledger, with a dated source and plan/account verification for each allowance. Keep account-specific unknowns unknown until checked.
- Alert at 70% and 85% of the design(cluster): egress strategy for 2 Gbps home fiber — budgets, tunnel topology, failure matrix #5030 operating envelope; at 85%, stop admitting optional bulk transfers and review before raising budgets. Do not disable critical control-plane/backup recovery traffic automatically.
- Validate recovery behavior after a counter reset/reboot and document gaps where provider APIs do not expose egress bytes.
Dependencies
Blocked by the new addressing/fabric plan in #1084 and by the #5030 design review. Independent of OCI signup (#4904).
Acceptance
A reproducible dashboard/report shows WAN bytes per direction and available SaaS counters; each series is labelled measured/estimated/unavailable; alert thresholds and operator actions are tested; the external subscription/usage ledger is updated with current account-specific allowance evidence.
No HashiCorp tooling; configuration and state must be declarative/recoverable.
Acceptance criteria
- An authoritative per-direction WAN byte counter at the edge is identified and monthly counters survive router reboots, with the counter source named on this issue.
- One monthly report exists in the usage/billing hub (or its source ledger) recording measured Cloudflare cache/origin bytes, R2 and Workers usage, and Turso sync if used — each value dated, sourced, and marked measured vs estimated, with plan allowances verified or explicitly left unknown.
- Alerts fire at 70% and 85% of the design(cluster): egress strategy for 2 Gbps home fiber — budgets, tunnel topology, failure matrix #5030 operating envelope, and the 85% response (stop admitting optional bulk transfers, review before raising budgets) is written into the report/runbook.
Activity
dezren39 commentedon Nov 4, 2026
Intake review
Related: Parent: #5030 (design(cluster): egress strategy). Related umbrella: #1090. Blocked-by in substance: #1084 (addressing/fabric plan).
Notes: Implementation slice of the #5030 egress design (which is in user/review): it cannot fully start until #1084 establishes the addressing/fabric plan and #5030's envelope is final — the scope text in the body says so itself. No merged PR resolves this (searched this run). Effort l: provider-metric plumbing across Cloudflare/R2/Workers/Turso plus edge counters and alerting; expect slicing (counters first, report second, alerts last). Risk: treating the 2 Gbps carrier rate as a data allowance — the body explicitly warns against it; acceptance keeps unverified allowances 'unknown'. Suggested approach: land the edge counter + monthly report skeleton with the provisional #5030 envelope, wire alerts once the envelope is confirmed.
— fleet monitor intake (protocol §5), 2026-11-03 18:12 CST run. Verified still live this run per the notes above.
dezren39 commentedon Nov 4, 2026
Agent note — task 9466 triaged
epic(cluster): network fabric— OPEN. impl(cluster): establish measured egress budgets and alerting #5203's scope says: after epic(cluster): network fabric #1084 establishes the new addressing/fabric plan, identify the WAN counter — the counter source cannot be named until that plan lands.design(cluster): egress strategy— OPEN, currently claimed/in-progress withuser/review. impl(cluster): establish measured egress budgets and alerting #5203's alerts are defined as percentages of the design(cluster): egress strategy for 2 Gbps home fiber — budgets, tunnel topology, failure matrix #5030 operating envelope, which does not exist yet.effort/l, not yet actionable. Fix path, in order: (1) design(cluster): egress strategy for 2 Gbps home fiber — budgets, tunnel topology, failure matrix #5030 design review closes with an operating envelope; (2) epic(cluster): network fabric #1084 addressing/fabric plan lands; (3) then implement per impl(cluster): establish measured egress budgets and alerting #5203 acceptance — edge per-direction WAN counter surviving reboots, monthly report in the usage/billing hub with measured/estimated/unavailable labelling, 70%/85% alerts with the 85% bulk-transfer stop written into the runbook.triaged; issue stays open.dezren39 commentedon Nov 5, 2026
🤖 Agent note · handed off
dezren39 commentedon Nov 5, 2026
🤖 Agent note · done
Re-triage findings (fresh reads, 2026-11-05):
epic(cluster): network fabric) — still OPEN. The WAN counter source cannot be named until its addressing/fabric plan lands.design(cluster): egress strategy) — still OPEN and inuser/review; the operating envelope that impl(cluster): establish measured egress budgets and alerting #5203's 70%/85% alerts are percentages of does not exist yet.effort/l, not yet actionable. No follow-up fix task enqueued: it would only park against the same two gates. Re-triage / enqueue the fix when design(cluster): egress strategy for 2 Gbps home fiber — budgets, tunnel topology, failure matrix #5030 and epic(cluster): network fabric #1084 are closed.triaged; issue impl(cluster): establish measured egress budgets and alerting #5203 stays open.