Skip to content

Raise the HTTP/1 line limit to 64 KiB and expose header limits on Transport - #1365

Merged
quinnj merged 4 commits into
masterfrom
fix/h1-max-line-bytes
Sep 17, 2026
Merged

quinnj merged 4 commits into
masterfrom
fix/h1-max-line-bytes

Conversation

@quinnj

@quinnj quinnj commented Sep 14, 2026

Copy link
Copy Markdown
Member

Problem

The HTTP/1 client failed any response containing a line longer than 8 KiB:

HTTP.ProtocolError: http protocol error: HTTP/1 line exceeds configured max_line_bytes

Header lines that long occur in practice (a 9,695-byte Content-Security-Policy was seen in the wild), curl and Python read such responses without complaint, and there was no way to raise HTTP.jl's limit: _HTTP1_DEFAULT_MAX_LINE_BYTES = 8 * 1024 was hard-coded in src/http1.jl, _read_transport_incoming_response in src/http_transport.jl took max_line_bytes/max_header_bytes with those defaults, both call sites passed nothing, and Transport had no keyword for either limit (HTTP.Transport(max_line_bytes = ...) was a MethodError).

Change

  • _HTTP1_DEFAULT_MAX_LINE_BYTES is now 64 KiB (Python's http.client _MAXLINE). The 1 MiB header-block limit (_HTTP1_DEFAULT_MAX_HEADER_BYTES) is unchanged and still bounds memory. The constant is shared with the server-side request parser (read_request), so the server's per-line limit for request lines and request header lines moves to 64 KiB as well; its max_header_bytes block limit (1 MiB default, configurable on Server) is untouched.
  • Transport gains max_line_bytes and max_header_bytes keywords (stored as fields, documented in the Transport docstring and the client guide). Both must be positive and max_line_bytes may not exceed max_header_bytes (a longer line can never be accepted past the block limit, so that combination is rejected as dead configuration). max_line_bytes defaults to min(64 KiB, max_header_bytes), so Transport(max_header_bytes = 16 * 1024) alone works and pulls the per-line limit down with it.
  • _read_transport_incoming_response reads both limits directly from its transport, including when it consumes informational heads and chunked trailers.
  • CHANGELOG entries under [Unreleased] (### Added for the keywords, ### Changed for the default bump).

Client(; ...) does not forward generic transport options (only local_addr folds into the default transport), so callers tune these via HTTP.Client(transport = HTTP.Transport(max_line_bytes = 128 * 1024)).

Tests

New tests in test/http_client_transport_tests.jl using a raw-TCP mock origin (skipped on Windows CI like the file's other raw-socket tests):

  • a 9,695-byte Content-Security-Policy header line succeeds with defaults; the same response fails with ProtocolError (_PROTOCOL_ERROR_LINE_TOO_LONG) under Transport(max_line_bytes = 8 * 1024);
  • a header line of exactly 64 KiB on the wire (the limit counts the CRLF) is accepted, one byte more is rejected with the same ProtocolError, and Transport(max_line_bytes = 128 * 1024) accepts it;
  • Transport(max_header_bytes = 4 * 1024) rejects a ten-line header block with _PROTOCOL_ERROR_HEADERS_TOO_LARGE while the default accepts it;
  • constructor validation: defaults, max_header_bytes-only lowering, zero/negative values and max_line_bytes > max_header_bytes throw ArgumentError.

Also verified end to end with the issue's reproducer through HTTP.get against a raw-socket origin: a 9,695-byte header line returns 200 (it failed before), a line over 64 KiB still raises the same ProtocolError, and the Transport keyword lowers and raises the limit as expected. Full Pkg.test run locally on Julia 1.12.6.

Fixes #1362

🤖 Generated with Claude Code

Review validation: added end-to-end checks for limits after 103 Early Hints and on chunked trailers. The transport and test-policy suites pass locally.

Co-authored by Codex

@codecov

codecov Bot commented Sep 14, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.72%. Comparing base (2edb99c) to head (e44e865).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1365      +/-   ##
==========================================
+ Coverage   89.70%   89.72%   +0.01%     
==========================================
  Files          31       31              
  Lines       12675    12680       +5     
==========================================
+ Hits        11370    11377       +7     
+ Misses       1305     1303       -2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

…nsport

The HTTP/1 client failed any response with a line longer than 8 KiB
(`ProtocolError: HTTP/1 line exceeds configured max_line_bytes`), and
nothing public could raise the limit: `_HTTP1_DEFAULT_MAX_LINE_BYTES` was
hard-coded, `_read_transport_incoming_response` fell back to it at both
call sites, and `Transport` had no keyword for either limit. Header lines
that long occur in practice (a 9,695-byte Content-Security-Policy was
reported), and curl and Python accept them.

- Raise `_HTTP1_DEFAULT_MAX_LINE_BYTES` from 8 KiB to 64 KiB (Python's
  `http.client` `_MAXLINE`). The 1 MiB header-block limit is unchanged and
  still bounds memory. The constant is shared with the server-side request
  parser, so its per-line limit moves too.
- Add `max_line_bytes` and `max_header_bytes` keywords to `Transport`,
  validated positive with `max_line_bytes <= max_header_bytes`;
  `max_line_bytes` defaults to `min(64 KiB, max_header_bytes)`.
- Plumb the transport's limits into `_read_transport_incoming_response`
  at both call sites instead of the module constants.
- Document the knobs in the `Transport` docstring, the client guide, and
  the changelog; add raw-TCP mock-origin tests for the 9,695-byte line,
  the boundary at the new default, lowering/raising via the keyword, the
  header-block limit, and constructor validation.

Fixes #1362

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@quinnj
quinnj force-pushed the fix/h1-max-line-bytes branch from fc2bbcd to a1d25b6 Compare September 14, 2026 21:42
@quinnj
quinnj merged commit 174cac1 into master Sep 17, 2026
10 checks passed
@quinnj
quinnj deleted the fix/h1-max-line-bytes branch September 17, 2026 14:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

HTTP/1 client rejects response header lines over 8 KiB, with no way to raise the limit

1 participant