Repository navigation
Description
fleet-proxy(http): response reader trusts peer-declared sizes (unbounded preallocation) and silently drops the body on a malformed Content-Length, desyncing the pooled connection
Problem
HttpConn::request in pkgs/fleet/crates/fleet-proxy/src/http.rs — the pooled HTTP/1.1 client the broker's serve side uses for tunneled services (serve.rs pools these connections per target port) — reads responses with none of the bounds the rest of the crate applies, and misparses malformed length headers instead of failing:
- Unbounded preallocation from the declared length. At
http.rs:147, aContent-Lengthbody is allocated in full before a single body byte arrives:data.resize(len, 0). There is no cap anywhere in the reader. The chunked path has the same shape athttp.rs:190(out.resize(start + size, 0)per chunk, parsed straight from the peer's hex chunk-size line, with no cumulative bound either). The status line and header lines are read with unboundedread_line(http.rs:98,:118). Contrast the same crate's own discipline:fleet-proxy-protorefuses frames overMAX_FRAME_BYTESbefore allocating, andtunnel.rscaps a CONNECT response at 65536 bytes. - A malformed
Content-Lengthsilently becomes "no body". Athttp.rs:133,content_length = value.parse().ok()— an unparsable value (e.g.Content-Length: abc) turns intoNone, so the response is treated as a bodiless keep-alive response: the real body bytes are left in the stream, the caller gets an empty body with a success status, and the connection staysreusableand returns to the pool, where the leftover bytes desync the next response parsed on it. DuplicateContent-Lengthheaders with differing values get the same silent treatment (last one wins). Every other malformed element in this reader (status line, header line, chunk size) is aMalformederror that marks the connection non-reusable — the length header is the one exception.
Evidence (verified this run)
Laptop probe crate ~/git/fleet/probe-proxy-http (path dep on the real fleet-proxy crate @ main 9cf2688), canned loopback servers:
- Invalid CL: response
HTTP/1.1 200 OK+Content-Length: abc+ bodyhello→requestreturnsOk((200, ""))— body silently dropped — andreusable() == true. A second request on the same connection returnsOk((200, "hi"))from a correctly-formed second response: the leftoverhellobytes were consumed as the start of the next status line, which only parses by accident (helloHTTP/1.1 200 OKsplits so token 1 is200). A body whose first token differs would corrupt the next caller's response instead. - Duplicate differing CLs:
Content-Length: 5thenContent-Length: 0, bodyhello→ same result:Ok((200, "")), reusable, silent desync of the next response. - Preallocation: response headers declaring
Content-Length: 536870912(512 MiB) with zero body bytes sent and the socket closed → the request errors on EOF only after allocating the full declared body: probe processVmHWMmeasured 526,616 KiB. A peer can declare up tousize::MAX; the allocation attempt itself is the failure (abort/OOM in the broker daemon), no payload ever needs to arrive.
The response peer is a service the broker tunnels to — not fully trusted input: a buggy or compromised local service, or anything that can answer on a tunneled port, can OOM the broker or poison pooled connections for unrelated later requests.
Why it matters
The broker is long-lived infrastructure; its pooled connections are shared across requests and callers. Today a single malformed response can (a) crash the daemon by allocation, or (b) silently return an empty body to one caller and shift the byte stream under the next caller — a correctness failure that surfaces far from its cause and is nearly impossible to diagnose from the second caller's symptoms.
Acceptance criteria
- An unparsable
Content-Length, or duplicateContent-Lengthheaders whose values differ, is aMalformederror (like the other header defects) and marks the connection non-reusable. - Response bodies have a documented maximum size (a named constant, sized for the JSON payloads this client carries, e.g. aligned with an existing crate ceiling): a declared
Content-Lengthabove it, a chunk size above it, and a chunked body whose cumulative size exceeds it all fail with an error before allocating the declared amount; the connection is marked non-reusable. - Status/header/chunk-size lines have a bounded read (no unbounded
read_lineaccumulation from a peer that never sends a newline). - Regression tests (canned loopback responses, following the crate's existing integration-test pattern): invalid CL →
Malformed+ not reusable; conflicting duplicate CLs →Malformed; oversize declared CL → error with no outsized allocation; oversize chunk → error; a well-formed CL response and a well-formed chunked response still round-trip and stay reusable.
Activity
developing-today commentedon Nov 4, 2026
CODE-1379
dezren39 commentedon Nov 4, 2026
🤖 Agent note · filed
dezren39 commentedon Nov 4, 2026
🤖 Agent note · picked up
dezren39 commentedon Nov 4, 2026
🤖 Agent note · done
MAX_RESPONSE_BODY_BYTES(64 MiB) cap enforced pre-allocation on CL + chunked (per-chunk and cumulative) + close-delimited paths, bounded line reads (MAX_LINE_BYTES64 KiB), 9 pinning loopback testsfleet-proxy(http): no-body responses (HEAD/204/304) and 1xx handling (#…