Add PrewarmedConnection to open the signalling connection ahead of connect by cnderrauber · Pull Request #1667 · livekit/rust-sdks

cnderrauber · 2026-11-12T09:52:38Z

Summary

Adds PrewarmedConnection: a TCP (+TLS for wss/https) connection to the LiveKit server opened before Room::connect. Pass it as RoomOptions::prewarmed_connection and the first signal connect only does the WebSocket upgrade, skipping TCP and TLS setup.

livekit-net: PrewarmedConnection::{connect, is_alive, close}. Used at most once. A watcher task notices a server-side close. If the connection is dead, points at another host, or its upgrade doesn't finish within 1.5 s (reached only on a silent drop with no FIN or RST; a close or reset falls back at once), connect falls back to a fresh dial; HTTP error statuses (e.g. 401) are returned as-is. Skipped when HTTP(S)_PROXY is set or a host transport is registered. Works with native-tls, rustls native roots and rustls webpki roots.
livekit-signaling / livekit: SignalOptions::prewarmed_connection / RoomOptions::prewarmed_connection. Only the first open_transport attempt uses it; v0 fallback, region failover and resume dial fresh.
livekit-ffi:
- PrewarmRequest → PrewarmCallback with an owned handle.
- PrewarmedConnectionAliveRequest.
- RoomOptions.prewarmed_connection_handle.
- Dropping the handle closes the socket.

Why

Agents can open the connection in idle job processes, so a dispatched job doesn't pay TCP+TLS round trips. Benchmark (dispatch → agent joined, network conditioner, WARP on, wss): 631 → 400 ms p50 for connect, 1069 → 836 ms dispatch→joined.

Tests

livekit-net/tests/prewarm.rs: upgrade over the warm stream with no second dial; used once across clones; 401 returned; a failed upgrade returns None.
Unit tests: endpoint normalization; a server close marks the connection dead; drop and close close the stream; a different host isn't used; a silent server times out.
livekit-ffi: prewarmed_connection_handle_lifecycle, 20× prewarm → alive → drop, with the server seeing EOF each time, plus the failure path.

Follow-ups

feat(rtc): add PrewarmedConnection python-sdks#937: rtc.PrewarmedConnection.
feat: prewarm the room connection in idle job processes agents#8522: keeps a connection ready in idle job processes.

🤖 Generated with Claude Code

…nnect A PrewarmedConnection is a TCP (+TLS for wss/https) connection to the LiveKit server opened before Room::connect. Passed as RoomOptions::prewarmed_connection, the first signal connect only performs the WebSocket upgrade over it, with the same URL, query and headers as a normal connect. - livekit-net: PrewarmedConnection::{connect, is_alive, close}. Used at most once; a watcher task notices a server close or reset. A dead connection, one opened for another host, or an upgrade that fails or exceeds 1.5s (reached only on a silent drop) falls back to a fresh dial; HTTP error statuses are returned as-is. Not used with HTTP(S)_PROXY or a host-registered transport. - livekit-signaling / livekit: SignalOptions/RoomOptions::prewarmed_connection, used by the first open_transport attempt only. - livekit-ffi: PrewarmRequest/PrewarmCallback returning an owned handle, PrewarmedConnectionAliveRequest, and RoomOptions.prewarmed_connection_handle. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

github-actions · 2026-11-12T09:53:00Z

Changeset ✓

This PR includes a changeset covering all affected packages:

Package	Bump
`livekit`	`minor`
`livekit-api`	`patch`
`livekit-capture`	`patch`
`livekit-ffi`	`minor`
`livekit-net`	`minor`
`livekit-signaling`	`minor`
`livekit-telemetry`	`patch`
`livekit-token-source`	`patch`

devin-ai-integration

Devin Review found 4 potential issues.

devin-ai-integration · 2026-11-12T09:54:34Z

SignalStream::connect

+        match tokio::time::timeout(timeout.min(UPGRADE_TIMEOUT), upgrade).await {
+            Ok(Ok((ws, _))) => {
+                log::debug!("using pre-warmed connection to {}", crate::redact_url(&parsed));
+                Ok(Some(WsConnectResult {
+                    connection: Arc::new(super::connection::NativeConnection::new(ws)),
+                }))
+            }
+            Ok(Err(e @ tungstenite::Error::Http(_))) => Err(super::proxy::map_ws_err(e)),
+            Ok(Err(e)) => {
+                log::warn!("pre-warmed connection failed, dialing a fresh one: {e}");
+                Ok(None)
+            }
+            Err(_) => {
+                log::warn!("pre-warmed connection timed out, dialing a fresh one");
+                Ok(None)


🔴 Slow upgrade sends duplicate joins

When a live server answers an upgrade after 1.5 seconds, Warm::upgrade abandons its request and asks for a fresh dial. Both requests can reach the server, creating two joins for one connection attempt.

Learn more

A WebSocket upgrade request has already been written to the server when the 1.5-second deadline expires. The server can process that join even though the client drops its waiting future. SignalStream::connect immediately opens a second connection on Ok(None), sending the same join again. The fixed limit also applies to a healthy but slow server, not just a silently dropped socket.

Example: A server receives a warm /rtc/v1 upgrade and takes 1.6 seconds to answer. The client abandons it at 1.5 seconds and sends a second upgrade with the same token and identity; the server can process both joins.

Recommended fix: Do not retry a possibly accepted join solely because the WebSocket response is slow. Use a deadline consistent with the connection attempt, or distinguish a known-dead stream from an uncertain, already-sent upgrade.

Was this helpful? React with 👍 or 👎 to provide feedback.

The answer costs 1 rtt so it should be less than 1.5s in most case. Even if two join reach the server, the server will pick a winner and kick/close another just like a normal repeat join.

- Give up opening a pre-warmed connection after 5 s, so a server that stalls TCP or TLS cannot leave the prewarm callback pending forever. - Release the prewarm handle when its callback cannot be delivered because the FFI server was disposed meanwhile. - Use one deadline for the pre-warmed attempt and the fresh dial after it, so a connect attempt never exceeds connect_timeout. - Bump the crates that depend on the changed ones in the changeset. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

xianshijing-lk

Reviewed head 101f340a. Three PR findings reproduced with targeted regression checks, plus one explicitly scoped shared FFI follow-up. The delayed-upgrade check demonstrates an actual DuplicateIdentity disconnection after fresh fallback has connected.

xianshijing-lk · 2026-11-12T17:34:31Z

+
+        let request = super::build_request(&parsed, headers)?;
+        let upgrade = tokio_tungstenite::client_async_with_config(request, stream, None);
+        match tokio::time::timeout(timeout.min(UPGRADE_TIMEOUT), upgrade).await {


[P1] An abandoned warm upgrade can disconnect the successful fallback

Timing out the upgrade does not stop its join request from being processed. With a proxy delaying the original request by 2.2 seconds, fresh fallback connects successfully, then the original request arrives and disconnects that room with DuplicateIdentity.

Please prevent an abandoned attempt from superseding the fallback connection. Keepalive or a different timeout value alone cannot guarantee this: the original TCP connection can be healthy while the request is delayed.

xianshijing-lk · 2026-11-12T17:34:31Z

-        .await
-        .map_err(|_| SignalError::Timeout("signal connection timed out".into()))??
-        .connection;
+        // One deadline covers the pre-warmed attempt and any fresh dial after it.


[P2] Preserve a fresh-dial budget when the configured timeout is short

When connect_timeout <= UPGRADE_TIMEOUT, the warm attempt can consume the entire deadline, leaving effectively no time for fresh fallback.

I reproduced this with an 800 ms timeout: cold connect succeeds, but supplying a stalled prewarm makes it fail. With the default five-second budget, fallback succeeds.

Please reserve time for fallback or skip the warm attempt when the budget is too short. Address this together with the late-join race; simply shortening the warm timeout makes that race more likely.

xianshijing-lk · 2026-11-12T17:34:31Z

+        if take_tx.send(reply_tx).is_err() {
+            return Ok(None);
+        }
+        let Ok(stream) = reply_rx.await else { return Ok(None) };


[P2] Include watcher handoff in the upgrade deadline

reply_rx.await runs outside the timeout wrapper. If the warm connection was created on a current-thread runtime that is no longer being driven, consuming it on another runtime waits indefinitely for the watcher.

A 20 ms upgrade remained pending after 200 ms in my test. SignalStream::connect has no outer timeout around this operation either.

Please apply an absolute deadline across both handoff and upgrade, leaving the remaining budget available for fallback.

xianshijing-lk · 2026-11-12T17:34:31Z

+        Err(err) => (proto::prewarm_callback::Message::Error(err.to_string()), None),
+    };
+    let sent =
+        server.send_event(proto::PrewarmCallback { async_id, message: Some(message) }.into());


[P2, shared FFI follow-up] Reject completions from a disposed setup

This is a pre-existing shared FFI issue in FfiServer::setup / send_event (room_apis/server/mod.rs:156,203) that also affects this new prewarm task. After dispose followed by setup, an old completion sees the replacement configuration and delivers its callback there. The prewarm reproduction also leaves an unsolicited handle in the map.

Please capture the setup generation when an async request starts and check it when registering result handles and delivering callbacks. This should be shared lifecycle handling, rather than a prewarm-only workaround.

cnderrauber requested review from jhugman, ladvoc and lukasIO as code owners November 12, 2026 09:52

generated protobuf

09a9427

This was referenced Nov 12, 2026

feat(rtc): add PrewarmedConnection livekit/python-sdks#937

Closed

feat: prewarm the room connection in idle job processes livekit/agents#8522

Closed

devin-ai-integration Bot reviewed Nov 12, 2026

View reviewed changes

xianshijing-lk reviewed Nov 12, 2026

View reviewed changes

cnderrauber closed this Nov 13, 2026

Navigation Menu

Repository navigation

Add PrewarmedConnection to open the signalling connection ahead of connect - #1667

Add PrewarmedConnection to open the signalling connection ahead of connect#1667
cnderrauber wants to merge 3 commits into
mainfrom
jie/prewarm-connection

cnderrauber commented Nov 12, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Nov 12, 2026 •

edited

Loading

Uh oh!

devin-ai-integration Bot left a comment

Uh oh!

Uh oh!

Uh oh!

devin-ai-integration Bot Nov 12, 2026 •

edited

Loading

Uh oh!

cnderrauber Nov 12, 2026

Uh oh!

Uh oh!

xianshijing-lk left a comment

Uh oh!

xianshijing-lk Nov 12, 2026

Uh oh!

xianshijing-lk Nov 12, 2026

Uh oh!

xianshijing-lk Nov 12, 2026

Uh oh!

xianshijing-lk Nov 12, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

cnderrauber commented Nov 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Why

Tests

Follow-ups

Uh oh!

github-actions Bot commented Nov 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changeset ✓

Uh oh!

devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

devin-ai-integration Bot Nov 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cnderrauber Nov 12, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

xianshijing-lk left a comment

Choose a reason for hiding this comment

Uh oh!

xianshijing-lk Nov 12, 2026

Choose a reason for hiding this comment

Uh oh!

xianshijing-lk Nov 12, 2026

Choose a reason for hiding this comment

Uh oh!

xianshijing-lk Nov 12, 2026

Choose a reason for hiding this comment

Uh oh!

xianshijing-lk Nov 12, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

cnderrauber commented Nov 12, 2026 •

edited

Loading

github-actions Bot commented Nov 12, 2026 •

edited

Loading

devin-ai-integration Bot Nov 12, 2026 •

edited

Loading