Skip to content

feat(observability): export standalone queue, ownership and execution signals #1252

Closed

Description

@joryirving

Why

Dispatch needs to reveal eligible work, waiting assignments, hand-outs, stale ownership, PR-fix settlement and sync/grooming health independently of Courier. Implement the signal and state catalogue in #1237 / PR #1239.

Work

  • Implement a bounded, read-only snapshot/exporter for queue depth/age, eligible and withheld work, assignment vs observed active execution, Lease/AgentWork, PR-fix hand-out/ack/reclaim, operational PARKED vs human BLOCKED, and sync/scheduler/groomer freshness.
  • Separate task execution signals from groomer/heartbeat/legacy AgentRuns via the accepted allowlist; don't infer a task pickup from a row written only at report completion.
  • Use the actual persisted IssueSyncRun statuses (running, completed, failed) and an absent-series/no-success baseline. Honor fresh-sync/eligibility guards when claiming a fleet has stopped.
  • Expose the inputs required for aged specific unadmitted assignments and PR-fix overdue unacked hand-outs. A continuously nonzero aggregate count in a busy queue is not evidence that one item is stranded.
  • Windowed database-derived failure metrics are gauges, not monotonic counters. Express window-specific counts clearly; never apply counter-rate/delta rules that lose window events.
  • Bounded enum labels only. No repo, issue, PR, worker identity, arbitrary error text or secrets in metric dimensions. No coupling to Courier phases or home-ops/Grafana installation.

Acceptance

  • Healthy idle, busy steady churn, genuinely stranded assignment, no first successful sync, failing discovery, inactive worker, and held dependency produce distinguishable observed/unknown states.
  • Tests prove a fleet with overlapping but promptly claimed assignments is not reported as 'aged stranded'; an actual aged assignment is. Windowed audit-failure arrivals at equal counts are still observable.
  • Requesting/exporting metrics is read-only; no claim release, requeue, status mutation, credential changes or mandatory backend.

Dependencies

Design complete as of 2026-10-18: #1237 is closed and #1239 merged. The implementation is READY at P3. No technical prerequisite remains; Dispatch's own priority/aging scheduler decides ordering relative to other ready tasks. Coordinate with #1202/#1203 lifecycle-event foundation; do not duplicate it. Dashboards and executable alerts are separate.

Activity

  1. added
    status/backlogNot ready for worker pickup yet.
    and removed
    status/readyReady for Dispatch worker pickup.
    on Oct 19, 2026
  2. its-saffron commented on Oct 19, 2026

    @its-saffron
    Contributor

    Grooming note: this is moving off status/ready pending verification, not because the ask is unclear — it is well bounded with concrete acceptance criteria. The design-complete claim in the Dependencies section (#1237 closed, #1239 merged) and the state of #1202/#1203 could not be confirmed from evidence captured this pass, and no standalone exporter implementation was verifiable on main. Once the design catalogue and the current metrics surface (e.g. docs/observability.md, prisma/schema.prisma, the API route layer) are captured, it can be re-groomed to ready.

  3. added and removed on Oct 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions