Repository navigation
fleet-proxy(http): response reader caps every line but nothing caps repetition — endless 1xx heads, header lines, or trailer lines hang an exchange (and its pooled tunnel) forever #6509
Description
Problem
fleet-proxy's HTTP response reader (pkgs/fleet/crates/fleet-proxy/src/http.rs) bounds every unit a peer can send — one line is capped at MAX_LINE_BYTES (:52, enforced in read_line_bounded :143), a declared body is capped at MAX_RESPONSE_BODY_BYTES before allocation — but nothing bounds repetition. Three loops consume peer-controlled units forever:
- Interim 1xx heads —
request()(:215) loopsread_response_head()andcontinues on every 1xx (except 101) with no count limit. - Header lines —
read_response_head()(:282, loop at :305) reads header lines until a blank line with no line-count limit. - Chunked trailer lines —
read_chunked()(:355, trailer loop :369) reads trailer lines until a blank line with no line-count limit.
A laptop service that is stuck, buggy, or hostile can therefore hang one broker exchange indefinitely: every unit arrives promptly, so the tunnel's 660 s per-read timeout (tunnel.rs TUNNEL_READ_TIMEOUT, applied per read syscall) never fires — each successful read starts a fresh window. The exchange holds its pooled tunnel (per-port pool, serve.rs) and broker thread the whole time; repeated requests to the same misbehaving service can pin the whole pool (default 50) one exchange at a time, with no error ever surfaced.
This is the one peer-trust gap left in a module whose stated doctrine (#5579, module doc :16-27) is that peer-declared sizes are never trusted: sizes are capped, counts are not.
Evidence (verified this run)
Probe ~/git/fleet/probe-proxy-http on the laptop (real fleet-proxy crate via path dependency, http.rs byte-identical at laptop HEAD and origin/main 85c435e; loopback fake peer, HttpConn::request on a thread with a timed wait):
- Control — peer sends one
100 Continuehead then200 OKwithContent-Length: 2: RETURNED status=200 body_len=2 in ~202µs. flood1xx— peer streamsHTTP/1.1 100 Continue\r\n\r\nheads at 1 ms intervals: STILL BLOCKED after 4s (would run to the peer's 200,000-head budget and beyond).floodhdr— peer sendsHTTP/1.1 200 OK\r\nthen streamsX-Fill: y\r\nheader lines: STILL BLOCKED after 4s.floodtrailer— peer sends a chunked head +0\r\nthen streams trailer lines: STILL BLOCKED after 4s.
Dedup: #5579 (closed) capped declared sizes; #5921 (closed) made 1xx interim heads consumed rather than returned — it introduced loop 1 without a bound. No open issue covers repetition/count bounds in this reader (searched: "fleet-proxy 1xx", "interim response unbounded", "http.rs trailer").
Why it matters
The broker is the fleet's single transport chokepoint to laptop services. A hang here is silent and unbounded — not a slow error but a permanently pinned tunnel and thread per exchange — and the trigger is exactly the peer this module already treats as untrusted for sizes. The failure mode also defeats the pool's bounded-wait design: slots are never returned, discarded, or timed out.
Acceptance criteria
- Each repetition loop in the response reader has an explicit, named bound (interim-head count, header-line count per head, trailer-line count), sized generously above anything a legitimate laptop service sends, and exceeding it returns
HttpError::Malformedand marks the connection non-reusable — the same failure shape as the existing size caps. - The probe cases above become tests in
http.rs' loopback test style: a peer flooding 1xx heads / header lines / trailer lines errors out promptly; the control (one interim head, normal headers, normal trailers) still succeeds. - Legitimate behaviour is unchanged: fleet-proxy(http): response reader waits for a body that never comes on HEAD / 204 / 304, and treats 1xx as the final response #5921's HEAD/204/304/1xx semantics and fleet-proxy(http): response reader trusts peer-declared sizes (unbounded preallocation) and silently drops the body on a malformed Content-Length, desyncing the pooled connection #5579's size caps keep passing (
cargo test -p fleet-proxyon the laptop or CI). - The module doc's peer-trust paragraph names the count bounds alongside the size bounds.
Activity
developing-today commentedon Nov 6, 2026
CODE-1968
dezren39 commentedon Nov 6, 2026
🤖 Agent note · filed
dezren39 commentedon Nov 6, 2026
Lane Q (code-q) claimed task 11023 for issue #6509. Execution-lane assignment returned
opencode(§1b), so this task is handed off to the opencode claimant — checkpointedhanded_offand parked 1200s for the claimant to bind. Attempts 1/4.21 remaining items
dezren39 commentedon Nov 6, 2026
🤖 Agent note · handed off
dezren39 commentedon Nov 6, 2026
🤖 Agent note · design-architect pass 62 — adopted as Phase 2 of design #6582
files/designs/code-fleet-proxy-validated-bounded.mdin the keep-the-four-repos-healthy goal workspacehttp.rsresponse reader (interim-head, header-line, trailer-line loops). Independent of Phase 1 in codehttp.rsmodule doc's peer-trust paragraph grows from sizes to all four dimensions (type, range, arithmetic, count), per your acceptance's doc criterion; bound values are yours to set as named constants beside theMAX_*size constants, "generously above anything a legitimate laptop service sends" per your acceptance — no legitimate-count evidence exists in-tree for the design to pin numbers againstdezren39 commentedon Nov 6, 2026
🤖 Agent note · handed off