Teleoperate robots, run remote policies, and record HITL demonstrations with LiveKit Portal.
Quickstart · Documentation · Examples · Benchmark
LiveKit Portal connects robots to remote operators over the internet. It carries camera frames and robot state to the operator, and control actions back to the robot. It works with any robotics stack, using LiveKit Cloud or a self-hosted LiveKit server.
- Low-latency video. Built on LiveKit's WebRTC media stack. In our benchmark, the full capture-to-action loop takes 26 ms round trip within San Francisco, with Portal adding under 2 ms over plain LiveKit. Use Portal's metrics to measure the loop on your own setup, including policy inference.
- Control arbitration. Keep policies and human operators in the same session. The robot accepts actions from one active operator; transfer control with one call or let a supervisor coordinate handoffs. Control handoff.
- Synchronized observations. Receive camera frames and joint state together, matched by capture timestamp, so a remote policy can compute its next action. Observations.
- Shared robot time. Participants synchronize to the robot's clock, giving actions, recordings, and annotations a common timeline across machines. Time sync.
- Data recording sinks. Add an observer to record alongside any operator.
Write Rerun archives with
RrdSink, or implement a sink for your own storage format. Data recording. - Policy evaluation. Record inactive policies' actions for comparison, mark episodes with annotations, and measure observation-to-action latency. Action subscriptions and metrics.
- Video for models and people. Use WebRTC video for live viewing, PNG or RAW for exact RGB pixels, or MJPEG for independent frame compression. Frame video.
Imagine your robot has two operators: a policy running in the cloud and a human ready to take over. The policy misses a grasp. The human steps in, demonstrates a correction, then returns control. You want to record the entire session to improve the policy.
Building this directly on vanilla WebRTC, WebSockets, or Media over QUIC (MoQ) gives you transport primitives; the robot's control arbitration remains your application's job. You need to track participants, choose the active operator, propagate handoffs, and reject commands from inactive operators. That means maintaining shared session state in a coordination service or realtime database, and handling concurrent requests and reconnects.
You also need clock synchronization and a buffering strategy for streams arriving at different times. As participants join from different networks, you must keep observations aligned without letting a slow stream add excessive control latency.
A recorder that logs only the teleoperator's inputs misses the policy's actions. If it timestamps incoming data on arrival, network delays become part of the recording: a camera frame and a joint reading captured together appear to have happened at different times. Separate logs from each machine leave you to reconstruct the handoff across different clocks.
Portal puts the robot, both operators, and an independent recorder in one LiveKit room. The robot accepts actions from the active operator, and a single call transfers control. LiveKit distributes the active-operator state through participant attributes, so you do not need a separate database for that handoff. Participants synchronize to the robot's clock, so frames, state, actions, and episode annotations share a timeline. The recorder can capture both operators' actions, including what the policy proposed while the human was in control, with each action labeled by sender and control status.
That gives you the policy attempt, the human correction, and the context needed to compare them in one recording. Write it to a Rerun archive or your own sink, then choose how to align and select samples for training.
LiveKit provides the foundation: multi-participant rooms and WebRTC transport across NAT and changing network conditions, without a VPN or manual port forwarding. Portal adds observation matching, clock synchronization, control arbitration, and recording. The same code runs on LiveKit Cloud or a self-hosted server.
Start with one robot and one operator, then add a human or recorder as your application grows.
uv add livekit-portal
# or
pip install livekit-portalFollow the quickstart to run your first control loop without physical hardware. Then add control handoff and data recording. For an existing LeRobot application, use the LeRobot plugins.
|
Basic |
SO-101 |
For configuration, tuning, and troubleshooting, see the documentation. For methods and callbacks, see the API reference.
Use the benchmark example to measure latency, jitter, and the full capture-to-action loop on your network. It streams synthetic video and state and replies to each observation immediately. No hardware required.
The sample results below use H264 at 640×480 and 30 fps. The capture-to-action loop is a round trip (RTT), measured from capture on the robot until the operator's reply reaches the robot.
| Local LiveKit server | Within San Francisco | San Francisco ↔ Hanoi, Vietnam |
|---|---|---|
|
|
|
|
| Loop RTT: 6.84 ms p50 | Loop RTT: 26.0 ms p50 | Loop RTT: 264.5 ms p50 |
Each LiveKit Cloud result is the median of three interleaved runs. One-way values in the Cloud diagrams are estimates derived from round trips.
To measure what Portal adds over a real network, compare plain LiveKit SDK round trips with Portal round trips between the same machines. Portal added under 2 ms on a 19 ms LiveKit route within San Francisco, and stayed within ±3 ms of plain LiveKit on a 259 ms route from San Francisco to Hanoi. See Measuring overhead across regions for the method, full results, and delivery gaps observed on the cross-region route.
Apache-2.0. See LICENSE for details.
| LiveKit Ecosystem | |
|---|---|
| Agents SDKs | Python · Node.js |
| LiveKit SDKs | Browser · Swift · Android · Flutter · React Native · Rust · Node.js · Python · Unity · Unity (WebGL) · ESP32 · C++ |
| Starter Apps | Python Agent · TypeScript Agent · React App · SwiftUI App · Android App · Flutter App · React Native App · Web Embed |
| UI Components | React · Android Compose · SwiftUI · Flutter |
| Server APIs | Node.js · Golang · Ruby · Java/Kotlin · Python · Rust · PHP (community) · .NET (community) |
| Resources | Docs · Docs MCP Server · CLI · LiveKit Cloud |
| LiveKit Server OSS | LiveKit server · Egress · Ingress · SIP |
| Community | Developer Community · Slack · X · YouTube |