perf: cut socket-layer per-event overhead (header cache, route LRU, deferred event routing)

[Alek99]

All Submissions:

• Have you followed the guidelines stated in CONTRIBUTING.md file?
• Have you checked to ensure there aren't any other open Pull Requests for the desired changed?

Type of change

• Bug fix (non-breaking change which fixes an issue)

Changes To Core Features:

• Have you added an explanation of what your changes do and why you'd like us to include them?
• Have you written new tests for your core changes, as applicable?

Note: the orjson wire-encoding change was dropped from this PR per @masenf's request, to be taken through @benedikt-bartscher's more complete PR. This PR now covers only the header cache, route LRU, and lock-hold changes.

Three per-event costs on the websocket hot path (ENG-10098):

1. Per-sid header cache (reflex/app.py) — on_event re-decoded every ASGI header from bytes and re-parsed x-forwarded-for on every event, though headers are fixed for a connection's lifetime. Now decoded once per sid, evicted on disconnect. A fresh dict is handed out per event so router_data mutations can't leak into the cache.

2. Route resolution LRU (reflex/route.py) — the router ran a linear regex scan over all routes per event for a path that rarely changes. The per-router get_route closure is now lru_cache(maxsize=1024); the route set is fixed for a router's lifetime (it was already baked in at closure creation), so no staleness is introduced.

3. Deferred chained-event routing (base_state_processor.py) — the final chained-event routing (frontend event encode+emit, backend event enqueue) ran while holding the state lock; it now runs after release. Deltas intentionally still emit under the lock: delta values reference live state containers (get_value unwraps proxies without copying), and e.g. emit_lost_and_found awaits redis before pickling, so a post-release emit could serialize data mutated by the next lock holder (thanks @masenf / the Codex review for flagging this — the delta-emit deferral from an earlier iteration was reverted). Chained-event payloads are proxy-detached at creation, so their deferred routing is safe. Per-token ordering is unchanged (the next event for a token dispatches only after the current task finishes, deferred routing included).

Measured results (Python 3.13, real before/after code with stubbed config):

cost	before	after	speedup
header decode per event (16 headers)	2.96 µs	0.24 µs	12.5x
route match per 4 paths (69-route app)	29.7 µs	0.32 µs	93x

Existing processor/route/app unit tests plus the integration suites cover the behavior.

Linear: ENG-10098

🤖 Generated with Claude Code

https://claude.ai/code/session_01Sns7EuBPYvfGCxRSRZ1bUx

[greptile-apps]

Greptile Summary

This PR reduces websocket event-path overhead. The main changes are:

• Cached decoded client headers and client IP per socket session.
• Cached route resolution with a per-router LRU.
• Deferred final chained-event routing until after the state lock is released.
• Added performance news entries for the changed event path.

Confidence Score: 5/5

This looks safe to merge.

• No blocking issues found in the changed code.

Important Files Changed

Filename	Overview
packages/reflex-base/src/reflex_base/event/processor/base_state_processor.py	Moves final chained-event routing outside the state lock while keeping delta emission inline.
reflex/app.py	Adds a per-session cache for decoded headers and client IP, with disconnect cleanup and per-event header copies.
reflex/route.py	Adds a per-router LRU cache around route matching.

_{Reviews (7): Last reviewed commit: "refactor: drop orjson wire encoder from ..." | Re-trigger Greptile}

[codspeed]

Merging this PR will not alter performance

✅ 27 untouched benchmarks
⏩ 8 skipped benchmarks¹

---

_{Comparing claude/reflex-perf-optimizations-21kg8y-socket (ab21ed3) with claude/reflex-perf-optimizations-01l7a3-eng-10095 (f9e4010)}

Footnotes

1. 8 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[linear-code]

ENG-10098

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bbf451db35

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[benedikt-bartscher]

related to my orjson pr #6116

[masenf]

can we decouple the orjson changes from this patch, i'd rather take those through @benedikt-bartscher more complete PR

[Alek99]

Done — dropped the orjson wire encoder (and its parity tests, benchmark, and dependency edge) in 0c8d27e, so it can go through @benedikt-bartscher's PR instead. This PR now scopes to just the per-connection header cache, route LRU, and deferring chained-event routing past the state lock (delta emit stays under the lock, as discussed above).

---

Generated by Claude Code

[FarhanAliRaza]

Closing as superseded. I checked each part of this PR's own diff (pr-6743-base..pr-6744) against current main (50cb0ce) and measured the only part that is still new.

What main already has

Part	Status on main
Header / client-IP cache	Done in #7068: _static_router_data[sid] is built once in on_connect and popped in on_disconnect.
Route LRU	Done in #7025 and simplified in #7153: @functools.lru_cache(maxsize=4096) on get_route.
Deferred chained-event routing	Not on main.

The diff no longer applies to main: reflex/app.py, reflex/route.py, and base_state_processor.py all conflict.

Deferred routing: A/B

Setup

• Builds: main against main plus only the deferral, ported by hand. Each build ran in its own isolated uv env. The port passes the event, app, and state unit suites.
• Server: reflex run --env prod --backend-only.
• Client: a python-socketio client doing full websocket round trips, after hydrate.
• Scenarios: counter increment, list append, nested mutation, substate + computed var, dynamic route, a 3-step backend chain (chain3), a handler that returns rx.call_script (script), and a 32-client mixed load test.
• Sampling: 7 interleaved rounds, 400 samples per scenario per round.
• Reading the tables: each cell is the median of the per-round medians in µs, with the per-round range in brackets. The last column counts the rounds the deferral won. incr and child are controls, because the patch does not touch them.

In-memory state manager

scenario	main	+ deferral	Δ	rounds won
incr (control)	398 [359–437]	389 [374–440]	−2.5%	2/7
child (control)	366 [345–391]	359 [333–380]	−2.1%	5/7
chain3	917 [878–958]	895 [879–955]	−2.5%	4/7
script	556 [507–606]	538 [521–553]	−3.1%	4/7
load (events/s)	2944 [2862–3054]	2952 [2870–3040]	+0.3%	2/7

All changes are as large as the changes in the control rows, so they are noise.

Redis state manager (default settings, oplock off)

scenario	main	+ deferral	Δ	rounds won
incr (control)	1654 [1603–1702]	1667 [1587–1708]	+0.8%	3/7
chain3	4939 [4799–5126]	5008 [4765–5184]	+1.4%	4/7
script	1764 [1738–1892]	2002 [1922–2085]	+13.5%	0/7
load (events/s)	2387	2366	−0.9%	2/7

Lock hold time

This was measured with a timing wrapper around modify_state_with_links, over 3 rounds. The deferral does shorten the hold:

manager	chained events	handlers that return call_script
in-memory	~185 → ~162 µs	~270 → ~190 µs
Redis	~1195 → ~1160 µs	~1385 → ~1170 µs

The shorter hold does not show up end to end, because same-token lock contention is not the bottleneck. The deferral also moves frontend events (call_script, redirects, toasts) out of the lock. Under Redis, those events now leave only after the state commit and unlock, so the client receives them about 240 µs later. They can also arrive after a later event's delta.

A smaller version that defers only backend enqueue is the chain3 row. It shows no gain in either manager, so there is nothing left to land.

One more note: this diff also deletes the "invalidation re-checks the return type" assertions in tests/units/reflex_base/vars/test_base.py.