Skip to content

perf(client): streamed text no longer republishes thread queue and checkpoints - #27425

Open
juliusmarminge wants to merge 4 commits into
mainfrom
fix/issue-27325
Open

juliusmarminge wants to merge 4 commits into
mainfrom
fix/issue-27325

Conversation

@juliusmarminge

Copy link
Copy Markdown
Member

Problem

While a turn streams, providers that send message.updated on every tick (ACP agents including the dev Replay provider, Cursor, OpenCode) give the client's thread projection a new messages array each tick. Two selectors in packages/client-runtime/src/state/threadDetail.ts, the thread queue workflow and the checkpoint summaries, re-derived and published a new value with identical contents on every tick. Every subscriber that reads only these selectors (the queue sheet on mobile, the diff panel and chat view on web) re-rendered on every streamed update for nothing. The issue measured streaming on iPad with the Replay provider and names this per-update thread publication as one of the causes.

Change

Both selectors now keep the value they last published when the new value is equal, and publish a new one only when something they derive actually changes.

  • Queue workflow: compares activeRun, the four flags, and each queued entry's run and message by identity. A queued entry's text, attachments and context all come from those two.
  • Checkpoint summaries: if the checkpoints array is the same, the summaries are the same, except for which assistant message each one points at. So a new value is published only when the checkpoints change or a summary's assistantMessageId does.

Scope and approval

Fixes #27325

Triaged via the automated issue flow. This PR covers the per-update republication of these two selectors. The issue's other items are left for follow-up: the work-row shimmer, full markdown re-parse on each update, Liquid Glass compositing, the route-level thread state subscription, and idle display links.

Verification

  • packages/client-runtime/src/state/threadDetail.test.ts, derived thread state through a streamed message: the tests apply streamed message.updated text ticks and check that the queue workflow and checkpoint summaries keep their identity. They also check that a queued message edit, a new assistant message for a checkpointed run, and a new checkpoint each still publish a new value. Removing the assistantMessageId comparison makes the checkpoint test fail.
    • Before (tests on fa4e4677c0, fix not applied):
      FAIL … > keeps the queue workflow through streamed text and follows a queued message edit
      AssertionError: expected { activeRun: null, …(5) } to be { activeRun: null, …(5) } // Object.is equality
      FAIL … > keeps checkpoint summaries through streamed text and follows a new checkpoint
      AssertionError: expected [ { …(9) } ] to be [ { …(9) } ] // Object.is equality
      Tests  2 failed | 14 passed (16)
      
    • After (this branch, threadDetail.test.ts and threadWorkflows.test.ts): Tests 38 passed (38).
  • vp run check:changed passes.
  • No visible surface changes, so there are no captures. These tests show the selectors no longer republish. They do not measure power. On mobile, the composer hook still subscribes to the whole thread, so it still re-renders on each update.
  • cannot-verify ios-device: the issue's power measurement needs a physical iPad with battery telemetry. The mechanism is shared client-runtime logic and is covered by the tests above.

Claude Opus 5.5 (Claude Code) and GPT-6.1 Sol (Codex), via the T3 Code issue flow.

🤖 Generated with Claude Code

juliusmarminge and others added 4 commits October 30, 2026 06:45
…checkpoint state

These tests reproduce #27325. A streamed assistant
message.updated that only grows the message text makes queueWorkflowAtom
and checkpointSummariesAtom publish a new value. Neither derived value
changed, but the composer and queue subscribers wake on every tick.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eckpoints

Providers that stream assistant text through message.updated replace the
messages array on every tick. The queue workflow and checkpoint summary
selectors re-derived a fresh value each time, waking the composer and
queue control. Both now keep their previous value when what they derive
is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…sage

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:M 30-99 changed lines (additions + deletions). labels Oct 30, 2026
@macroscopeapp

macroscopeapp Bot commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Approved at 093810b

Macroscope's review found this PR approvable — This change avoids redundant queue and checkpoint subscriber updates while assistant text streams. Its effects are confined to client-side derived-state publication, with no changes to product defaults or persisted data.

You can add or adjust custom eligibility rules. Learn more.

@coderabbitai

coderabbitai Bot commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 97e35bc and 093810b.

Files selected for processing: 2
  • packages/client-runtime/src/state/threadDetail.test.ts
  • packages/client-runtime/src/state/threadDetail.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 4 remain after this review.

Walkthrough

Queue workflow and checkpoint summary atoms now retain cached values when their derived state meets unchanged-state checks. Tests cover identity retention during streamed updates and updates after queued-message edits, new assistant messages, and captured checkpoints.

Changes: Thread state identity

Layer / File(s) Summary
Retain unchanged derived state
packages/client-runtime/src/state/threadDetail.ts, packages/client-runtime/src/state/threadDetail.test.ts
The queue workflow atom compares active runs, workflow flags, and queued run and message identities before replacing its cached value. The checkpoint summary atom retains its array when checkpoint references and assistant message IDs remain unchanged. Tests cover identity retention and state updates.

Priority: Normal

Estimated code review effort: 3 (Moderate) | ~15 minutes

Change: Refactor · Severity of issue fixed: Medium

Suggested reviewers: t3dotgg, macroscopeapp

Merge Risk: Minimal · up to 09381

No merge-blocking risk was found.

Pre-merge checks: Passed 4
Check name Status Explanation
Title check Passed The title "perf(client): streamed text no longer republishes thread queue and checkpoints" matches the main change. The PR makes the queue workflow and checkpoint summary selectors keep their last pub…
Description check Passed The description follows the template. It has Problem, Change, Scope and approval, and Verification sections. The problem and change are clearly explained and match the code summary. Scope links the tr…
Linked Issues check Passed Direct issue [#27325] identifies per-update state publication as a streaming cost and supports focused, non-visible fixes. This PR changes the queue workflow selector and checkpoint-summary selector t…
Out of Scope Changes check Passed The changed source is limited to the two selectors in packages/client-runtime/src/state/threadDetail.ts. The added tests in packages/client-runtime/src/state/threadDetail.test.ts verify those sele…
Full details: Title check

Explanation

The title "perf(client): streamed text no longer republishes thread queue and checkpoints" matches the main change. The PR makes the queue workflow and checkpoint summary selectors keep their last published value when derived contents are unchanged, so streamed text updates no longer republish them. The title is specific and follows the conventional commit format.

Full details: Description check

Explanation

The description follows the template. It has Problem, Change, Scope and approval, and Verification sections. The problem and change are clearly explained and match the code summary. Scope links the triaged issue #27325 and states what is left for follow-up. Verification lists focused tests with before and after results, notes that no visible surface changed, and states what could not be verified (iPad power measurement). The template asks for an approval comment or discussion link. The description says the issue came from the automated issue flow but does not link a maintainer approval comment. It also does not explicitly name the approval comment as the template requires. The template also asks for the model and harness, and the description includes them.

Full details: Linked Issues check

Explanation

Direct issue [#27325] identifies per-update state publication as a streaming cost and supports focused, non-visible fixes. This PR changes the queue workflow selector and checkpoint-summary selector to retain their prior values when derived contents are unchanged. The queue comparison covers the active run, four flags, and queued run/message identities. The checkpoint comparison covers checkpoint identity and each summary's assistantMessageId. Tests cover streamed updates and required changes for queued edits, new assistant messages, and new checkpoints. The issue does not require this PR to fix every listed contributor or provide physical-device power telemetry.

Full details: Out of Scope Changes check

Explanation

The changed source is limited to the two selectors in packages/client-runtime/src/state/threadDetail.ts. The added tests in packages/client-runtime/src/state/threadDetail.test.ts verify those selector changes. No unrelated product behavior, shimmer, markdown parsing, Liquid Glass, route subscription, or display-link changes are present in the reviewed diff.

Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@dbxe

dbxe commented Oct 30, 2026 •

Copy link
Copy Markdown

Note

🤖 Claude Opus 5.5 responding on behalf of Daniel Briskin

This covers one slice of #27325 (the queue and checkpoint selectors), so could the body say Refs #27325 instead of Fixes? The larger items are still open: the work-row shimmer, re-parsing the whole message on every update, and the route-level thread subscription. I'm running this branch through the same iPad bench tonight and will post the number here.

@dbxe

dbxe commented Oct 30, 2026

Copy link
Copy Markdown

Note

🤖 Claude Opus 5.5 responding on behalf of Daniel Briskin

Bench result: no measurable power change while a turn streams, which is what I expected for two selectors.

  • Setup: the issue's iPad bench, streaming phase only, 2 ABBA/BAAB blocks. The PR's threadDetail.ts diff was applied onto the issue's baseline dac8b11ddc, because the bench backend is pinned there.
  • Result: −5 / −11 mW per block against the baseline, about 1059 mW. That reads same: the bench resolves about 25 mW.

That doesn't argue against merging. It just means the streaming cost lives in the other items. I'm measuring candidates for the markdown re-parse and the route and composer subscriptions next.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M 30-99 changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: iOS app draws ~2.8x the power of a minimal Flutter client while a turn streams

2 participants