Skip to content

feat: adopt Go 1.27 (encoding/json/v2, jsontext, generic methods) - #18

Merged
mpyw merged 2 commits into
mainfrom
feat/go-1.27
Aug 22, 2026
Merged

mpyw merged 2 commits into
mainfrom
feat/go-1.27

Conversation

@mpyw

@mpyw mpyw commented Aug 22, 2026 •

Copy link
Copy Markdown
Owner

Raises the toolchain to Go 1.27 and takes up the parts of 1.25–1.27 that this project can actually use.

encoding/json/v2 — request bodies

internal/server/body/json.go decodes with v2 defaults, which follow RFC 7493. The parsed map becomes the SQL bind parameters, so v1 silently keeping the last of a duplicate member meant {"id":1,"id":2} could override a parameter a caller believed it had set. Duplicate members and invalid UTF-8 are now rejected as 400.

encoding/json/v2 — responses

Responses are this project's public API, so marshalOptions opts out of v2's stricter/tidier defaults one by one rather than accepting them wholesale. internal/server/responder_test.go pins every row below, so a failure means a v2 default leaked into the wire format:

v1 v2 default this PR
map key order sorted insertion sorted (Deterministic)
non-UTF-8 column value U+FFFD error → 500 U+FFFD (AllowInvalidUTF8)
nil slice null [] null (FormatNilSliceAsNull)
nil map null {} null (FormatNilMapAsNull)
time.Time RFC 3339 RFC 3339 RFC 3339
[]byte column base64 base64 base64
NaN / Inf error error error → 500
< > & < … literal literal

With those four options, v2 output is byte-identical to v1 for every type a SQL driver can hand back. The last row is the only remaining difference and it is deliberate: responses are served as application/json, where v1's HTML escaping bought nothing.

Two of these would have been silent regressions: without AllowInvalidUTF8, rows that serve fine today (latin1/binary columns) become a 500; the nil-map case is reachable through a post-transform handing back a nil map for an empty type: one result.

Mock JSON/JSONL sources decode with v2 too, so a duplicate key in fixture data is now a config error instead of a silent last-wins.

encoding/json/jsontext — JSONL streaming

JSONL parsing streams through a jsontext.Decoder instead of reading whole lines with bufio.Scanner. This removes the 64KiB per-line limit that made any large JSONL row fail with token too long; a regression test covers a 128KiB row.

Generic methods

internal/mock/filter.go and internal/mock/value_parser.go each carried their own copy of the pooled-goja-VM dance — sync.Pool wiring, VM construction, timeout watchdog, interrupt unwrapping. Both now use js.PooledVM, whose Call method takes the result type as a parameter:

p.Call(map[string]any{"ctx": ctx}, goja.Value.ToBoolean, row, input)  // bool
p.Call(nil, goja.Value.Export, value)                                 // any

The type parameter belongs on the method rather than a free function because the conversion has to run while the runtime is still checked out — a goja.Value must not outlive its runtime's return to the pool.

Bug found while consolidating

goja.Runtime.Interrupt leaves its flag set if it lands after the script already finished, and time.Timer.Stop does not wait for an in-flight timer function. A filter row finishing right at the 5s deadline could therefore return a poisoned runtime to the pool, and every later row on that runtime failed with a spurious timeout.

runWithTimeout now disarms the watchdog under a mutex and clears the flag before returning, and a runtime that really did time out is dropped instead of pooled. TestRunWithTimeout_ClearsLateInterrupt fails without the fix (verified by reverting it).

Other new API applied

API Since Where
errors.AsType[E] 1.26 all 5 errors.As sites — 4 used the (*T)(nil) idiom just to get a typed target
http.Server.MaxHeaderValueCount 1.27 100, with MaxHeaderBytes 64KiB and a ReadHeaderTimeout
url.URL.Clone 1.27 js.NewRequest, so the request JS sees is a snapshot
strings.CutLast 1.27 config.getParentPath
testing/synctest.Test 1.25 the delay tests
maps.Copy — CSV row copy (via go fix)

Only the header phase is bounded by a timeout, so large uploads and slow queries are not cut off.

Checked and not applied

The remaining new API in 1.25–1.27 was cross-referenced against the official api/go1.{25,26,27}.txt lists, filtered to the packages this repo imports directly. Nothing else fits — sync.WaitGroup.Go (no WaitGroup), database/sql.ConvertAssign (sqlx MapScan does the conversion), httptest.NewTestServer (tests call handlers directly), bytes.CutLast, bytes.Buffer.Peek, slog.NewMultiHandler, os.Root methods.

One deliberate omission: net/http.NewCrossOriginProtection (1.25) would add Sec-Fetch-Site-based CSRF protection, which is genuinely relevant for a proxy whose POST/PUT/PATCH/DELETE endpoints run SQL. But it is a policy decision that interacts with the configurable permissive CORS, not a mechanical toolchain update — worth its own PR if you want it.

Tests

  • decodeJSON[T] replaces 79 hand-rolled decode blocks across the e2e suite. Every assertion routed through it now also asserts the response is well-formed RFC 7493 JSON.
  • The delay tests run under testing/synctest: they assert the exact configured delay instead of a tolerance window, and no longer sleep (0.00s per subtest, and the flaky < 30ms assertion is gone).
  • New: internal/server/responder_test.go (the compatibility table above, plus the NaN 500 path and that Error's lo.Must cannot panic on a non-UTF-8 driver message), the strict decode paths, the 128KiB JSONL row, and the interrupt fix.

Verified

  • go build ./..., go vet ./..., gofmt -l (Go 1.27.0) — clean
  • go test ./... and go test -race ./... — all pass
  • golangci-lint 2.13.1 (built with go1.27.0) — 0 issues

Notes for review

  • golangci-lint must be ≥ v2.13.0 to parse generic methods (go1.27 support landed in v2.13.0, 2026-08-19). golangci-lint-action@v9 resolves to the latest release, so CI is fine unpinned — say the word if you'd rather pin it.
  • CI's test matrix drops to ['1.27']; encoding/json/v2 and generic methods rule out older toolchains. README/CLAUDE.md updated.
  • GOEXPERIMENT=nojsonv2 will no longer build this project. That opt-out is temporary upstream.
  • gofmt from a Go ≤1.26 toolchain cannot parse internal/js/pool.go (generic method). Contributors need a 1.27 toolchain locally.

🤖 Generated with Claude Code

Raise the toolchain to Go 1.27 and take up the parts of it that this
project can actually use.

encoding/json/v2

  Request bodies (internal/server/body/json.go) now decode with v2
  defaults, which follow RFC 7493. The parsed map becomes the SQL bind
  parameters, so v1 silently taking the last of a duplicate member let
  `{"id":1,"id":2}` override a parameter a caller believed it had set.
  Duplicate members and invalid UTF-8 are now rejected as 400.

  Responses (internal/server/responder.go) marshal with
  Deterministic(true) and AllowInvalidUTF8(true) so the observable
  output stays as it was under v1: map keys sorted, non-UTF-8 column
  values rendered as U+FFFD rather than failing the whole response.
  The one visible change is that <, > and & are no longer escaped -
  responses are served as application/json, where v1's escaping bought
  nothing.

  Mock JSON/JSONL sources decode with v2 as well, so a duplicate key in
  fixture data is now a config error instead of a silent last-wins.

encoding/json/jsontext

  JSONL parsing streams through a jsontext.Decoder instead of reading
  whole lines with bufio.Scanner. This removes the 64KiB per-line limit
  that made any large JSONL row fail with "token too long"; a
  regression test covers a 128KiB row.

Generic methods

  internal/mock/filter.go and internal/mock/value_parser.go each carried
  their own copy of the pooled-goja-VM dance: sync.Pool wiring, VM
  construction, timeout watchdog, interrupt unwrapping. Both now use
  js.PooledVM, whose Call method takes the result type as a parameter:

      p.Call(map[string]any{"ctx": ctx}, goja.Value.ToBoolean, row, input)
      p.Call(nil, goja.Value.Export, value)

  The type parameter belongs on the method rather than a free function
  because the conversion has to run while the runtime is still checked
  out - a goja.Value must not outlive its runtime's return to the pool.

  Fixes a latent bug found while consolidating: goja's Interrupt leaves
  its flag set if it lands after the script already finished, and
  time.Timer.Stop does not wait for an in-flight timer function. A
  filter row that finished right at the 5s deadline could therefore
  return a poisoned runtime to the pool, and every later row on that
  runtime failed with a spurious timeout. runWithTimeout now disarms the
  watchdog under a mutex and clears the flag, and a runtime that really
  did time out is dropped instead of pooled.

Other 1.27 API

  - http.Server: MaxHeaderValueCount and MaxHeaderBytes, plus a
    ReadHeaderTimeout. Only the header phase is bounded, so large
    uploads and slow queries are not cut off.
  - url.URL.Clone in js.NewRequest, so the request JS sees is a snapshot.
  - strings.CutLast in config.getParentPath.
  - maps.Copy in the CSV row copy (via go fix).

Tests

  - decodeJSON[T] replaces 79 hand-rolled decode blocks across the e2e
    suite. Every assertion routed through it now also asserts the
    response is well-formed RFC 7493 JSON.
  - The delay tests run under testing/synctest, so they assert the exact
    configured delay instead of a tolerance window, and no longer sleep.
  - New coverage for the strict decode paths, the large JSONL row, and
    the interrupt fix.

Note: golangci-lint must be v2.13.0 or later to parse generic methods.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@codecov

codecov Bot commented Aug 22, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.90909% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.90%. Comparing base (7bc1246) to head (74ac70b).

Files with missing lines Patch % Lines
internal/cli/commands/app.go 0.00% 5 Missing ⚠️
internal/js/pool.go 93.54% 4 Missing ⚠️
internal/mock/json.go 92.30% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main      #18      +/-   ##
==========================================
+ Coverage   76.55%   82.90%   +6.35%     
==========================================
  Files          34       35       +1     
  Lines        2013     1685     -328     
==========================================
- Hits         1541     1397     -144     
+ Misses        346      288      -58     
+ Partials      126        0     -126     
Files with missing lines Coverage Δ
internal/config/config.go 86.66% <100.00%> (+3.57%) ⬆️
internal/config/errors.go 94.68% <100.00%> (+0.98%) ⬆️
internal/js/apply.go 90.36% <100.00%> (+5.04%) ⬆️
internal/js/error.go 88.46% <100.00%> (+3.16%) ⬆️
internal/js/request.go 100.00% <100.00%> (ø)
internal/mock/csv.go 93.33% <100.00%> (+5.83%) ⬆️
internal/mock/filter.go 77.14% <100.00%> (+26.54%) ⬆️
internal/mock/value_parser.go 75.00% <100.00%> (+29.34%) ⬆️
internal/server/body/json.go 100.00% <ø> (ø)
internal/server/handler.go 89.53% <100.00%> (+0.31%) ⬆️
... and 4 more

... and 21 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

errors.AsType (Go 1.26)

  Replaces all five errors.As call sites. Four of them used the
  `(*T)(nil)` declare-in-condition idiom purely to get a typed target;
  AsType returns the value directly, so the type being matched now reads
  at the call site instead of in a throwaway variable.

encoding/json/v2: nil slice and nil map

  A v1 behavior difference missed in the previous commit: v2 renders a
  nil slice as `[]` and a nil map as `{}`, where v1 rendered both as
  `null`. Reachable through a post-transform that hands back a nil map
  for an empty `type: one` result. marshalOptions now sets
  FormatNilSliceAsNull and FormatNilMapAsNull.

  With those two added, v2 output is byte-identical to v1 for every type
  a SQL driver can hand back - time.Time, []byte, int64, float64, bool,
  string, nil, and maps/slices of them. The only remaining difference is
  the deliberate one: <, > and & are no longer escaped.

  internal/server/responder_test.go pins that contract. Each case is a
  v2 default we opt out of, so a failure means a v2 default leaked into
  the wire format. It also covers the NaN/Inf 500 path and that
  Error's lo.Must cannot panic on a non-UTF-8 driver message.

The remaining new API in Go 1.25-1.27 was checked against the official
api/go1.{25,26,27}.txt lists, filtered to the packages this repo imports
directly. Nothing else applies; net/http.CrossOriginProtection is the
one deliberate omission - it is a CSRF policy decision, not a mechanical
update, and would conflict with the configurable permissive CORS.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@mpyw
mpyw merged commit c5f6694 into main Aug 22, 2026
6 checks passed
@mpyw
mpyw deleted the feat/go-1.27 branch August 22, 2026 01:17
mpyw pushed a commit that referenced this pull request Aug 22, 2026
Picks up the ssh security fixes in 0.47.0-0.52.0: the source-address
critical option bypass, the infinite loop from integer overflow on large
channel writes, strict limits on DSA key parameters, rejection of RSA
keys with excessively large moduli, user presence verification for
security keys, and several deadlock/nil-panic fixes.

Rebased onto main by hand to resolve a go.mod/go.sum conflict. Two notes
on what changed versus the original Dependabot commit:

  - The range is 0.50.0 -> 0.52.0, not 0.46.0 -> 0.52.0. #17 moved
    x/crypto to 0.50.0 in the meantime.
  - Dependabot had to raise the `go` directive to 1.25.0, because
    x/crypto 0.52.0 requires it. That was the CI failure: on the then
    Go 1.23/1.24 matrix the go command auto-switched toolchains and the
    coverage step died with `go: no such tool "covdata"`. main is on
    1.27.0 since #18, so the directive is left alone and the failure
    is gone.

x/text 0.36.0 -> 0.37.0 comes along transitively via x/net 0.54.0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
mpyw pushed a commit that referenced this pull request Aug 22, 2026
Picks up the ssh security fixes in 0.47.0-0.52.0: the source-address
critical option bypass, the infinite loop from integer overflow on large
channel writes, strict limits on DSA key parameters, rejection of RSA
keys with excessively large moduli, user presence verification for
security keys, and several deadlock/nil-panic fixes.

Rebased onto main by hand to resolve a go.mod/go.sum conflict. Two notes
on what changed versus the original Dependabot commit:

  - The range is 0.50.0 -> 0.52.0, not 0.46.0 -> 0.52.0. #17 moved
    x/crypto to 0.50.0 in the meantime.
  - Dependabot had to raise the `go` directive to 1.25.0, because
    x/crypto 0.52.0 requires it. That was the CI failure: on the then
    Go 1.23/1.24 matrix the go command auto-switched toolchains and the
    coverage step died with `go: no such tool "covdata"`. main is on
    1.27.0 since #18, so the directive is left alone and the failure
    is gone.

x/text 0.36.0 -> 0.37.0 comes along transitively via x/net 0.54.0.

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant