Repository navigation
feat(events): retain frontend notices and reset expired cursors - #331
Merged
Merged
Conversation
Second half of the v1.0 ADR 0006 port. frontend_event only carries REST
invalidation notices, but it grew without bound. A background thread
owned by the local Workflow Authority now trims it even when nobody is
subscribed. It keeps the last 24 hours and at most 10,000 notices by
default (BasicConfig.frontend_event_*, overridable through UNILABOS_
environment variables), checks every 60 s and deletes at most 1,000 rows
per transaction. Each pass deletes a contiguous prefix only: it stops at
the first notice that must stay, so out-of-order times never leave holes,
and it always keeps the latest notice as the watermark. The thread stops
before the shared runtime.db connection closes.
/api/v1/events reads the retention bounds and the page in one SQL
snapshot on every poll. A cursor behind the window gets stream.reset
{reason: cursor_expired} with the latest sequence as its id. A cursor
exactly at the window edge resumes normally. An explicit Last-Event-ID: 0
still replays the retained window on its first page.
Checked against a two-process dev backend started with
UNILABOS_BASICCONFIG_FRONTEND_EVENT_MAX_COUNT=5 and a 1 s interval: the
table settles at 5 notices, SSE watch keeps working, and SIGTERM shuts
down cleanly.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Second step of bringing the v1.0 line into dev; it finishes the ADR 0006 port started in #330.
frontend_eventonly carries REST invalidation notices but grew without bound. A background thread owned by the local Workflow Authority now trims it even when nobody is subscribed. Defaults: last 24 hours, at most 10,000 notices, a check every 60 s and at most 1,000 rows per transaction. They are set byBasicConfig.frontend_event_*and can be overridden throughUNILABOS_environment variables. Each pass deletes only a contiguous prefix: it stops at the first notice that must stay, so out-of-order times never leave holes, and the latest notice is always kept as the watermark. The thread stops before the sharedruntime.dbconnection closes./api/v1/eventsreads the retention bounds and the page in one SQL snapshot on every poll. A cursor behind the window getsstream.reset{reason: cursor_expired}with the latest sequence as its id. A cursor exactly at the window edge resumes normally, and an explicitLast-Event-ID: 0still replays the retained window. Clients need no change: OpenLabSite (Xuwznln/OpenLabSite#2) and the SDK watch from #330 already treat anystream.resetas a REST reload. Tests: newtests/server/test_workflow_events_retention.pycovers contiguous-prefix deletion, the count and age limits, watermark retention, bounded batches and the worker lifecycle.test_workflow_events_subscription.pygains expired, edge and zero-replay cases. Locallytests/server tests/protocol tests/appgive 664 passed; the 3test_driver_package_graphsfailures also fail on current dev, caused by demo packages installed into the shared venv. Checked against a two-process dev backend withUNILABOS_BASICCONFIG_FRONTEND_EVENT_MAX_COUNT=5and a 1 s interval: the table settles at 5 notices, SSE watch keeps working and SIGTERM shuts down cleanly.🤖 Generated with Claude Code