You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Since #1135 (commit a4e01eb, merged 2026-09-24), the Next.js integration often serves invalid __NEXT_DATA__ on Next.js Pages Router pages. This needs only [integrations.nextjs]; no other integration has to be enabled. The corrupted element holds one or more unrewritten pieces of the original JSON, followed by the complete rewritten copy, so JSON.parse fails.
The trigger is how lol_html hands script text to the rewriters. It delivers the text in fragments. If one fragment ends with _, or with another leading part of __next_f (__, __n, __ne, __nex, __next, __next_), that fragment and the next one are emitted raw.
This is not an edge case. lol_html 2.9.0 splits long script text into 1 KiB fragments, so a large __NEXT_DATA__ has hundreds of fragment boundaries. I served 10 public Pages Router pages through the Fastly adapter under Viceroy with only Next.js enabled:
At a4e01eb, 6 of the 10 came back with invalid __NEXT_DATA__.
The same mechanism affects inline scripts when [integrations.google_tag_manager] is also enabled. An inline script that contains a GTM or GA URL is emitted twice: first the unrewritten text, then GTM's rewritten copy.
Steps to reproduce
Add this test to mod tests in crates/trusted-server-core/src/integrations/nextjs/mod.rs, next to html_processor_rewrites_nextjs_script_when_enabled. The module already has create_test_settings, config_from_settings, json!, Cursor and the pipeline types in scope.
/// With only Next.js enabled, a `__NEXT_DATA__` fragment that ends right/// after `_` must not be emitted twice.#[test]fnnext_data_fragment_ending_in_underscore_is_emitted_once(){letmut settings = create_test_settings();
settings
.integrations.insert_config("nextjs",&json!({"enabled":true,"rewrite_attributes":["href","link","url"]}),).expect("should update nextjs config");let registry = IntegrationRegistry::with_plan(&settings,Arc::new(crate::auction::compile_auction_plan(&settings).expect("should compile auction plan"),),).expect("should create registry");let start_tag = r#"<script id="__NEXT_DATA__" type="application/json">"#;let prefix = format!(r#"<html><body>{start_tag}{{"props":{{"pageProps":{{"href":"https://origin.example.com/reviews","body":""#);let suffix = r#""},"__N_SSP":true},"page":"/reviews"}</script></body></html>"#;// Make the first 8 KiB input chunk end right after the first `_` of `__N_SSP`.let underscore = suffix.find('_').expect("should contain an underscore");let html = format!("{prefix}{}{suffix}","x".repeat(8192 - prefix.len() - underscore - 1));assert_eq!(&html[8191..8192],"_","first input chunk should end with an underscore");let processor = create_html_processor(config_from_settings(&settings,®istry));letmut pipeline = StreamingPipeline::new(PipelineConfig{input_compression:Compression::None,output_compression:Compression::None,chunk_size:8192,},
processor,);letmut output = Vec::new();
pipeline
.process(Cursor::new(html.as_bytes()),&mut output).expect("should process HTML");let processed = String::from_utf8(output).expect("should produce UTF-8");let start = processed
.find(start_tag).expect("should keep __NEXT_DATA__")
+ start_tag.len();let end = start
+ processed[start..].find("</script>").expect("should close __NEXT_DATA__");let data: serde_json::Value = serde_json::from_str(&processed[start..end]).unwrap_or_else(|error| panic!("should keep __NEXT_DATA__ valid JSON: {error}"));assert_eq!(
data["props"]["pageProps"]["href"],"https://test.example.com/reviews","should rewrite the origin href");assert_eq!(
processed.matches("__N_SSP").count(),1,"should emit the __NEXT_DATA__ text once");}
cargo test -p trusted-server-core --target <host-triple> --lib -- \
next_data_fragment_ending_in_underscore_is_emitted_once
---- integrations::nextjs::tests::next_data_fragment_ending_in_underscore_is_emitted_once stdout ----
should keep __NEXT_DATA__ valid JSON: trailing characters at line 1 column 8162
Here the output is the whole unrewritten payload, still holding https://origin.example.com/reviews, followed by the rewritten payload. The test passes at 4c6d26a.
Real pages on the Fastly adapter. I used the Viceroy harness from #1206, with an app config that enables only [integrations.nextjs]. The origin was a local HTTP server serving saved, uncompressed copies of 10 public Pages Router pages. I then parsed __NEXT_DATA__ from each response. At a4e01eb, 6 of the 10 responses were 200 OK with invalid __NEXT_DATA__; at 4c6d26a, all 10 were valid. I also inspected three of the broken pages through publisher::stream_publisher_body. In each, 2,048 to 8,192 bytes of unrewritten text, copied from the middle of the payload, sat in front of the complete copy.
With GTM also enabled, the inline-script variant looks like this. An 8,233-byte inline script declares a gtag URL and calls gtag('config','G-TEST',{send_page_view:false}), and its first 8 KiB input chunk ends right after send_. At a4e01eb the response carries the script text twice, raw copy first: window.dataLayer= and gtag('config' each appear 2 times. At 4c6d26a they appear once.
Expected behavior
__NEXT_DATA__ is emitted exactly once, as valid JSON, with the configured URL rewrites, wherever fragment boundaries fall. Inline scripts are emitted once.
Actual behavior
When a fragment of __NEXT_DATA__ ends with _ (or __, __n and so on), unrewritten pieces of the payload are emitted ahead of the complete rewritten copy. The element is no longer valid JSON. With GTM enabled, inline GTM or GA scripts can be emitted twice.
Root cause
lol_html fragments long script text into 1 KiB pieces. Cargo.lock pins lol_html 2.9.0. Its text decoder emits the text found in the first input chunk as one fragment. After that first non-final chunk it keeps a pending streaming decoder, which disables the fast path, so the rest of the text node is decoded through a 1,024-byte buffer (DEFAULT_BUFFER_LEN in src/rewritable_units/text_decoder.rs). Measured on a real 303,922-byte __NEXT_DATA__ with 8 KiB input chunks: 293 fragments. The first was 6,620 bytes, then came 290 pieces of 1,024 bytes, a 342-byte remainder and an empty final fragment.
Two Next.js rewriters handle every fragment of __NEXT_DATA__. They are NextJsNextDataRewriter (selector script#__NEXT_DATA__) and NextJsRscPlaceholderRewriter (selector script), registered in that order (nextjs/mod.rs:97-98). create_html_processor gives each one its own lol_html text handler, and each handler applies its action directly to the shared chunk (html_processor.rs:664-694):
NextJsNextDataRewriter buffers every intermediate fragment and returns RemoveNode (nextjs/script_rewriter.rs:89, through capture_fragment in nextjs/rsc_stream.rs:89-144). On the last fragment it returns Replace with the complete rewritten text.
Make body hold parser-aware and stream Next.js processing #1135 changed NextJsRscPlaceholderRewriter. For any script fragment that does not contain __next_f but ends with a proper prefix of it, the rewriter now holds that tail back and returns Replace(&content[..ready_length]) (nextjs/rsc_placeholders.rs:222-237). On the next fragment it returns Replace with the held tail plus the new content (:240-263).
In lol_html 2.9.0, remove() only sets a flag, and a removed chunk emits whatever replacement any handler stored. So the placeholder rewriter's Replace wins over the __NEXT_DATA__ rewriter's RemoveNode. Those fragments reach the output raw, and the complete rewritten copy follows on the last fragment.
Before #1135, the placeholder rewriter returned Keep for every intermediate fragment. The comment at rsc_placeholders.rs:57-62 in 4c6d26a reads: "Accumulation here would also risk corrupting non-RSC scripts that happen to be fragmented during streaming".
With GTM enabled, the same override hits GTM's own buffering. GTM returns RemoveNode for fragments of scripts that contain its hosts and emits the rewritten script on the last fragment (google_tag_manager.rs:1045-1077). The placeholder rewriter runs before GTM and replaces a fragment that ends in _, and GTM's remove() cannot cancel that replacement.
Impact
Affected deployments: any deployment with [integrations.nextjs] enabled that serves Pages Router pages. The integration is off by default.
What visitors get: the Next.js 14.2.33 client reads the element with JSON.parse(document.getElementById("__NEXT_DATA__").textContent) (next/dist/client/index.js), so invalid JSON throws during start-up. The server-rendered HTML is shown but never hydrates. Interactive components do nothing, and there is no client-side routing.
Origin leak: the raw pieces are not rewritten, so any origin URLs inside them also end up in the page source.
App Router pages: they have no __NEXT_DATA__ and are not affected by the Next.js-only variant. The repository's App Router fixture (html_processor.test.html) is served completely with only Next.js enabled.
GTM variant: it needs an inline script that contains a GTM or GA host, spans a fragment boundary, and has a fragment ending in _. Standard GTM and gtag snippets are short, and in my sample none was at risk. When it does happen, the script runs twice, or fails to parse if it declares top-level let or const bindings.
How often a fragment ends right after _. First, what I measured:
Fragment sizes. Fragments are about 1 KiB after the first input chunk (see Root cause).
Public pages. 10 public Pages Router pages. I fetched front pages from a list of 20 well-known sites and kept the ones with __NEXT_DATA__, so they are not a representative sample of publisher pages.
Core runs. I fed each page to publisher::buffer_publisher_response_async with only Next.js enabled, at 32 chunk alignments. Alignments were 17 to 7,984 leading newline bytes, 257 bytes apart, so each one also lands at a different offset in the 1 KiB fragment grid. Each alignment ran once as an identity body and once gzip-encoded.
Viceroy runs. One request per page, uncompressed, as fetched.
In total, 174 of 320 identity runs and 198 of 320 gzip runs broke at a4e01eb. At 4c6d26a, none of the 320 identity or 320 gzip runs broke.
Second, what I reasoned. With a boundary about every 1 KiB at a position unrelated to the content, a payload with u underscores has about u/1024 boundaries that land right after one. The chance that a page breaks is then about 1 − e^(−u/1024): 4% for page A, 45% for D, 95% for H and 99% for J. That matches the measured rates within sampling noise.
When Trusted Server reads a whole body (8 KiB chunks), the outcome is fixed for a given version of a page. When the origin body streams (Fastly), the first boundary follows network reads, so the same page can break on some requests and not on others. I did not measure that on real Fastly traffic.
Proposed fix
Preferred: compose script rewriters in core, as proposed in #1208. lol_html still matches selectors, and one text!("script") handler pipes each chunk through the matching rewriters in registration order and applies a single mutation. A fragment held by the __NEXT_DATA__ rewriter then reaches the placeholder rewriter as empty text, and nothing can re-emit it. This also fixes the GTM variant and #1206. I prototyped it in a scratch copy of a4e01eb:
The test above passes.
The real-page measurement dropped to 0/320 identity and 0/320 gzip.
All 10 pages were valid under Viceroy.
The GTM inline script was emitted once.
Every existing test in a native cargo test -p trusted-server-core --lib run passed.
A narrower stopgap is possible if a quick patch is needed before that change. __NEXT_DATA__ is JSON, never a flight script, so NextJsRscPlaceholderRewriter::rewrite can leave its fragments alone while the __NEXT_DATA__ rewriter is capturing. Right after the document state is locked (nextjs/rsc_placeholders.rs:186-189), add:
if !matches!(state.next_data,super::rsc_stream::FragmentState::Idle){returnScriptRewriteAction::Keep;}
In the prototype, this stopgap made the test above pass and dropped the real-page measurement to 0/320 for both encodings. Every existing test still passed. It does not fix the GTM variant, where the other buffering rewriter is GTM.
A test with GTM and Next.js enabled splits an inline script that contains a GTM URL right after an _, and the script is emitted once.
A test streams a large __NEXT_DATA__ with many underscores at several chunk sizes (for example 64, 1,000 and 8,192 bytes) and gets valid JSON every time.
Description
Since #1135 (commit a4e01eb, merged 2026-09-24), the Next.js integration often serves invalid
__NEXT_DATA__on Next.js Pages Router pages. This needs only[integrations.nextjs]; no other integration has to be enabled. The corrupted element holds one or more unrewritten pieces of the original JSON, followed by the complete rewritten copy, soJSON.parsefails.The trigger is how lol_html hands script text to the rewriters. It delivers the text in fragments. If one fragment ends with
_, or with another leading part of__next_f(__,__n,__ne,__nex,__next,__next_), that fragment and the next one are emitted raw.This is not an edge case. lol_html 2.9.0 splits long script text into 1 KiB fragments, so a large
__NEXT_DATA__has hundreds of fragment boundaries. I served 10 public Pages Router pages through the Fastly adapter under Viceroy with only Next.js enabled:__NEXT_DATA__.The same mechanism affects inline scripts when
[integrations.google_tag_manager]is also enabled. An inline script that contains a GTM or GA URL is emitted twice: first the unrewritten text, then GTM's rewritten copy.Steps to reproduce
Add this test to
mod testsincrates/trusted-server-core/src/integrations/nextjs/mod.rs, next tohtml_processor_rewrites_nextjs_script_when_enabled. The module already hascreate_test_settings,config_from_settings,json!,Cursorand the pipeline types in scope.Observed at a4e01eb:
Here the output is the whole unrewritten payload, still holding
https://origin.example.com/reviews, followed by the rewritten payload. The test passes at 4c6d26a.Real pages on the Fastly adapter. I used the Viceroy harness from #1206, with an app config that enables only
[integrations.nextjs]. The origin was a local HTTP server serving saved, uncompressed copies of 10 public Pages Router pages. I then parsed__NEXT_DATA__from each response. At a4e01eb, 6 of the 10 responses were200 OKwith invalid__NEXT_DATA__; at 4c6d26a, all 10 were valid. I also inspected three of the broken pages throughpublisher::stream_publisher_body. In each, 2,048 to 8,192 bytes of unrewritten text, copied from the middle of the payload, sat in front of the complete copy.With GTM also enabled, the inline-script variant looks like this. An 8,233-byte inline script declares a gtag URL and calls
gtag('config','G-TEST',{send_page_view:false}), and its first 8 KiB input chunk ends right aftersend_. At a4e01eb the response carries the script text twice, raw copy first:window.dataLayer=andgtag('config'each appear 2 times. At 4c6d26a they appear once.Expected behavior
__NEXT_DATA__is emitted exactly once, as valid JSON, with the configured URL rewrites, wherever fragment boundaries fall. Inline scripts are emitted once.Actual behavior
When a fragment of
__NEXT_DATA__ends with_(or__,__nand so on), unrewritten pieces of the payload are emitted ahead of the complete rewritten copy. The element is no longer valid JSON. With GTM enabled, inline GTM or GA scripts can be emitted twice.Root cause
lol_html fragments long script text into 1 KiB pieces. Cargo.lock pins lol_html 2.9.0. Its text decoder emits the text found in the first input chunk as one fragment. After that first non-final chunk it keeps a pending streaming decoder, which disables the fast path, so the rest of the text node is decoded through a 1,024-byte buffer (
DEFAULT_BUFFER_LENinsrc/rewritable_units/text_decoder.rs). Measured on a real 303,922-byte__NEXT_DATA__with 8 KiB input chunks: 293 fragments. The first was 6,620 bytes, then came 290 pieces of 1,024 bytes, a 342-byte remainder and an empty final fragment.Two Next.js rewriters handle every fragment of
__NEXT_DATA__. They areNextJsNextDataRewriter(selectorscript#__NEXT_DATA__) andNextJsRscPlaceholderRewriter(selectorscript), registered in that order (nextjs/mod.rs:97-98).create_html_processorgives each one its own lol_html text handler, and each handler applies its action directly to the shared chunk (html_processor.rs:664-694):NextJsNextDataRewriterbuffers every intermediate fragment and returnsRemoveNode(nextjs/script_rewriter.rs:89, throughcapture_fragmentinnextjs/rsc_stream.rs:89-144). On the last fragment it returnsReplacewith the complete rewritten text.NextJsRscPlaceholderRewriter. For any script fragment that does not contain__next_fbut ends with a proper prefix of it, the rewriter now holds that tail back and returnsReplace(&content[..ready_length])(nextjs/rsc_placeholders.rs:222-237). On the next fragment it returnsReplacewith the held tail plus the new content (:240-263).In lol_html 2.9.0,
remove()only sets a flag, and a removed chunk emits whatever replacement any handler stored. So the placeholder rewriter'sReplacewins over the__NEXT_DATA__rewriter'sRemoveNode. Those fragments reach the output raw, and the complete rewritten copy follows on the last fragment.Before #1135, the placeholder rewriter returned
Keepfor every intermediate fragment. The comment atrsc_placeholders.rs:57-62in 4c6d26a reads: "Accumulation here would also risk corrupting non-RSC scripts that happen to be fragmented during streaming".With GTM enabled, the same override hits GTM's own buffering. GTM returns
RemoveNodefor fragments of scripts that contain its hosts and emits the rewritten script on the last fragment (google_tag_manager.rs:1045-1077). The placeholder rewriter runs before GTM and replaces a fragment that ends in_, and GTM'sremove()cannot cancel that replacement.Impact
[integrations.nextjs]enabled that serves Pages Router pages. The integration is off by default.JSON.parse(document.getElementById("__NEXT_DATA__").textContent)(next/dist/client/index.js), so invalid JSON throws during start-up. The server-rendered HTML is shown but never hydrates. Interactive components do nothing, and there is no client-side routing.__NEXT_DATA__and are not affected by the Next.js-only variant. The repository's App Router fixture (html_processor.test.html) is served completely with only Next.js enabled._. Standard GTM and gtag snippets are short, and in my sample none was at risk. When it does happen, the script runs twice, or fails to parse if it declares top-levelletorconstbindings.How often a fragment ends right after
_. First, what I measured:__NEXT_DATA__, so they are not a representative sample of publisher pages.publisher::buffer_publisher_response_asyncwith only Next.js enabled, at 32 chunk alignments. Alignments were 17 to 7,984 leading newline bytes, 257 bytes apart, so each one also lands at a different offset in the 1 KiB fragment grid. Each alignment ran once as an identity body and once gzip-encoded.__NEXT_DATA__bytes_countIn total, 174 of 320 identity runs and 198 of 320 gzip runs broke at a4e01eb. At 4c6d26a, none of the 320 identity or 320 gzip runs broke.
Second, what I reasoned. With a boundary about every 1 KiB at a position unrelated to the content, a payload with
uunderscores has aboutu/1024boundaries that land right after one. The chance that a page breaks is then about1 − e^(−u/1024): 4% for page A, 45% for D, 95% for H and 99% for J. That matches the measured rates within sampling noise.When Trusted Server reads a whole body (8 KiB chunks), the outcome is fixed for a given version of a page. When the origin body streams (Fastly), the first boundary follows network reads, so the same page can break on some requests and not on others. I did not measure that on real Fastly traffic.
Proposed fix
Preferred: compose script rewriters in core, as proposed in #1208. lol_html still matches selectors, and one
text!("script")handler pipes each chunk through the matching rewriters in registration order and applies a single mutation. A fragment held by the__NEXT_DATA__rewriter then reaches the placeholder rewriter as empty text, and nothing can re-emit it. This also fixes the GTM variant and #1206. I prototyped it in a scratch copy of a4e01eb:cargo test -p trusted-server-core --librun passed.A narrower stopgap is possible if a quick patch is needed before that change.
__NEXT_DATA__is JSON, never a flight script, soNextJsRscPlaceholderRewriter::rewritecan leave its fragments alone while the__NEXT_DATA__rewriter is capturing. Right after the document state is locked (nextjs/rsc_placeholders.rs:186-189), add:In the prototype, this stopgap made the test above pass and dropped the real-page measurement to 0/320 for both encodings. Every existing test still passed. It does not fix the GTM variant, where the other buffering rewriter is GTM.
Done when
next_data_fragment_ending_in_underscore_is_emitted_oncepasses._, and the script is emitted once.__NEXT_DATA__with many underscores at several chunk sizes (for example 64, 1,000 and 8,192 bytes) and gets valid JSON every time.Affected area
HTML processing / JS injection
Version
main at a4e01eb
Related
NextJsRscPlaceholderRewriterto act on intermediate fragments of every<script>.__NEXT_DATA__rewriter. Its proposed fix also resolves this issue.text.replace()wins" for overlapping script rewriters.