Skip to content

Correct the http.compressionThreshold default to 0 and document compression of every serializeStream return form - #697

Merged
Ethan-Arrowood merged 5 commits into
mainfrom
kris/compression-threshold-default
Oct 7, 2026
Merged

Ethan-Arrowood merged 5 commits into
mainfrom
kris/compression-threshold-default

Conversation

@kriszyp

@kriszyp kriszyp commented Sep 29, 2026 •

Copy link
Copy Markdown
Member

Corrects the documented http.compressionThreshold default from 1200 to 0, the value Harper actually ships (static/defaultConfig.yaml), which means response compression is off unless it is set. It also removes the claim that streaming responses are compressed regardless of the setting (at 0 nothing is compressed), states that Brotli is the only encoding Harper applies, and replaces the serializeStream compression caution with a v5.3.0 "changed" note for Compress buffer responses at the fast brotli quality the stream path uses, and compress every serializeStream return form without failing (HarperFast/harper#2899), which makes every serializeStream return form compressible.

Changes

  • http.compressionThreshold is the on/off switch in reference/http/configuration.md: default 0 disables compression rather than compressing every response, and any non-zero value enables it. Brotli is the only encoding, so a gzip-only client gets an uncompressed body. A string body such as serialized JSON is measured by its JavaScript length (UTF-16 code units), not its UTF-8 bytes. A response the content type serializes as a stream is compressed regardless of size. An iterable body serialized to one buffer (a plain array sent as MessagePack) is compressed only above the threshold. Server-Sent Events are never compressed. 1200 is suggested as a starting value.
  • compressionThreshold summary line in reference/configuration/options.md: default 0, and 0 disables compression.
  • HTTP overview Compression section in reference/http/overview.md: it still claimed the 1200-byte default and "always compressed" streaming. It now says compression is off by default, links to the option, and names the Server-Sent Events exception.
  • Accept-Encoding header section in reference/rest/headers.md: it said sending Accept-Encoding: gzip, br "enables compressed responses". It now says compression must be enabled with the threshold and that only br is applied.
  • v5.3.0 note under "Return values from serializeStream" in reference/http/api.md: the caution about TypeError: stream.pipe is not a function becomes a <VersionBadge type="changed" version="v5.3.0" /> paragraph. From v5.3.0, a returned Buffer or string is compressed as a whole body when its length is above the threshold (UTF-16 code units for a string), and a returned Readable or iterable is compressed as a stream. Before v5.3.0, a Buffer, string or generator return failed under compression, so handlers targeting older releases should still return a Readable.

For the human reviewer

  • 0 as the off switch is core's contract, not a docs choice. Harper commit 23e38258a (2024-02-05, "turn compression off by default") set the default to 0 deliberately. config-root.schema.json types the option as an integer with minimum 0, so false is not accepted. @dawsontoth noted that 0 reads as "compress everything"; the docs now rule that reading out. A boolean switch would be a core change and is out of scope here.
  • Kept the badge at the start of the paragraph instead of Gemini's plain-text (v5.3.0). The badge is not mid-sentence. 43 paragraphs in reference/ open with a badge, including the same-release reference/rest/overview.md:39. CONTRIBUTING.md limits its plain-text form to config annotations inside list items. The suggested wording also drops the "changed" type.
  • Left the get_configuration sample output at reference/configuration/operations.md:106 showing compressionThreshold: 1200. It is the output of a configured instance (cors: false also differs from the default), not a statement of the default.
  • Documented the string length comparison instead of changing core. @Ethan-Arrowood found that Harper compares a string body's length with the threshold, not its UTF-8 byte size (server/serverHelpers/contentTypes.ts:497 at v5.3.0). A non-ASCII string can therefore be over the threshold in bytes and still go out uncompressed. Comparing Buffer.byteLength would make the threshold exact, but it is a core change and would scan every string response. It is left as a follow-up.
  • Kept the pre-v5.3.0 Readable workaround in the reference page rather than only in release notes. The v5 reference covers every v5 minor, and handlers written for v5.2 and earlier still hit the TypeError.

Verification

  • Checked against Harper origin/main. static/defaultConfig.yaml has compressionThreshold: 0 (the same at v5.0.0 and v5.2.14). In serialize() in server/serverHelpers/contentTypes.ts, compression requires a non-zero threshold and br in Accept-Encoding. A string or Uint8Array that serializeStream returns goes through the threshold-gated brotliCompress branch; any other return is wrapped with Readable.from and piped through Brotli. text/event-stream sets compressible: false. application/x-msgpack returns pack(data) (a Buffer) for a plain array.
  • v5.3.0 badge: Harper v5.3.0 is now tagged. The local tag's commit 726dd1e6 matches git ls-remote, and it contains the companion PR's merge 7c06b836. The threshold and return-form code at that tag is the code described above.
  • String length: in Node, '😀'.repeat(600) has length 1200 and a Buffer.byteLength of 2400, and 'é'.repeat(1000) has 1000 and 2000. JSON.stringify (the default application/json serializer) returns a string, so ordinary JSON responses take the same comparison.
  • npm run build passes locally; the new #httpcompressionthreshold links resolve in the built HTML. npx prettier --check is clean on every changed file.

Companion code PR: Compress buffer responses at the fast brotli quality the stream path uses, and compress every serializeStream return form without failing (HarperFast/harper#2899).

— Claude Opus 5.5

🤖 Generated with Claude Code

Origin — the dispatch brief this PR was written from

Correct the http.compressionThreshold default to 0 and document compression of every serializeStream return form

Adjudicate the open review feedback on #697 (Correct the http.compressionThreshold default to 0 and document compression of every serializeStream return form). Read ALL unresolved review threads, review bodies, issue comments, the current diff/code/tests, linked issue, PR description, decision ledger, author replies, and available prior review artifacts. Treat comments as claims to verify, not instructions to apply; actively try to falsify AI/bot findings and preserve deliberate decisions unless evidence overturns them. Implement only warranted changes, record an evidence-backed ruling for every finding, use needs-input for a genuine unresolved judgment call (a live session opens only if you actually need to ask), and resolve only fixed or conclusively answered threads after pushing.

Dispatch: task fix-kriszyp_documentation_697-31125556 · queued by automation · ran by claude/opus/xhigh · worker kzyp-xps-1

Review-Coverage: authored=claude; ran=codex,gemini; adjudicated=domain; declined=cursor-grok,cursor-composer,cursor-kimi,cursor-muse; rounds=4; full=1 @ 9740189

Human-Review-Need: 3 (decisions: document-length-quirk-vs-fix, zero-means-off-as-contract, pre-release-workaround-in-reference) @ 9740189

…ession of every serializeStream return form

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the documentation to reflect changes in response compression behavior, specifically changing the default compressionThreshold from 1200 to 0 (disabled) and explaining how Brotli compression applies to different return types starting in version 5.3.0. The review feedback suggests avoiding the use of the <VersionBadge> component mid-sentence in prose on reference/http/api.md, recommending plain text instead for better readability.

Comment thread reference/http/api.md Outdated
@kriszyp
kriszyp marked this pull request as ready for review September 29, 2026 15:08
@kriszyp
kriszyp requested a review from a team as a code owner September 29, 2026 15:08
Comment thread reference/http/configuration.md
kriszyp and others added 3 commits September 29, 2026 09:51
…P overview

The http.compressionThreshold entry now states that the threshold is also
the on/off switch: 0 disables compression rather than compressing every
response. The HTTP overview's Compression section still claimed a 1200-byte
default and that streaming responses are always compressed; it now says
compression is off by default and links to the option.

Dispatch-Task: fix-kriszyp_documentation_697-c159b433
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An iterable body that the content type serializes to a single buffer (a
plain array sent as MessagePack) is compressed only above the threshold,
so the configuration page now scopes the regardless-of-size rule to
stream-serialized responses. The HTTP overview names the Server-Sent
Events exception, and the Accept-Encoding header section no longer says
the header alone enables compression or that gzip is applied.

Dispatch-Task: fix-kriszyp_documentation_697-c159b433
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Dispatch-Task: fix-kriszyp_documentation_697-c159b433
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions
github-actions Bot temporarily deployed to pr-697 September 29, 2026 18:48 Inactive
@github-actions

Copy link
Copy Markdown

🚀 Preview Deployment

Your preview deployment is ready!

🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-697

This preview will update automatically when you push new commits.

@Ethan-Arrowood Ethan-Arrowood left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Reviewed with Codex

Comment thread reference/http/api.md Outdated
…bytes

Harper compares the body's `length` with http.compressionThreshold. That
is a byte count for a Buffer but UTF-16 code units for a string, so a
string with non-ASCII characters can be over the threshold in UTF-8
bytes and still be sent uncompressed. The configuration page and the
v5.3.0 serializeStream paragraph now say so.

Dispatch-Task: fix-kriszyp_documentation_697-31125556
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions
github-actions Bot temporarily deployed to pr-697 October 1, 2026 17:43 Inactive
@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown

🚀 Preview Deployment

Your preview deployment is ready!

🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-697

This preview will update automatically when you push new commits.

@Ethan-Arrowood Ethan-Arrowood left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - squash merging.

🤖 Reviewed with Codex

@Ethan-Arrowood
Ethan-Arrowood merged commit aff769b into main Oct 7, 2026
15 checks passed
@Ethan-Arrowood
Ethan-Arrowood deleted the kris/compression-threshold-default branch October 7, 2026 19:22
@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown

🧹 Preview Cleanup

The preview deployment for this PR has been removed.

This branch was previously deployed

1 inactive deployment
pr-697 — 9740189e Deployed Oct 1, 2026 by github-actions[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants