Repository navigation
refactor(core): use Effect's official Bun and Node SQLite clients - #60798
Conversation
There was a problem hiding this comment.
The change looks sound. On dev, the new Node test fails with UnknownError. With this PR it gets ConstraintError, WAL is on and the Drizzle transaction commits. I checked this by running the PR's test against both checkouts. The Drizzle, workerd, interrupt, bundle, database, migration and session tests pass (825 tests), and typecheck:tsc in packages/core is clean. @effect/sql-sqlite-node 4.1.0 is built on node:sqlite with no native addon or extra dependencies, so the Node build doesn't gain a native module. BEGIN IMMEDIATE applies only to writable clients, and as you note, no Bun or Node production code calls withTransaction directly.
Public API is unchanged. Only internal #sqlite modules in packages/core are affected, plus the dependency changes.
Two small notes, neither blocking:
- The Bun case in
sqlite-node.test.tsalso passes ondevunder Bun 1.4.2, including the read-only reopen. On this Bun version it doesn't catch the regression described in the PR. - Nothing passes
timeouttosqliteLayer(database.tsonly passesfilename), so thetimeout→busyTimeoutalias insqlite.node.tsis unused. The mapping itself is correct.
Related: #53879
There was a problem hiding this comment.
Both earlier notes are resolved. The new commit removes the unused timeout alias from sqlite.node.ts, which now passes its config straight to SqliteClient.layer. The description now says the Bun read-only case only catches the regression on Bun 1.4.4 and later, which matches what I saw on 1.4.2. The Node, bundle and database tests pass on the new head, and typecheck:tsc in packages/core is clean. Public API is unchanged.
There was a problem hiding this comment.
The new commit adds tests and fixes a comment. The added Drizzle typed insert().returning() and select() checks pass on Node, including a working query after a constraint failure. The new tests passed in 5 consecutive runs, and typecheck:tsc in packages/core is clean. Public API is unchanged.
One suggestion (inline): the copied interrupt sweep in the Node fixture could be dropped, to keep that test small.
|
|
||
| const typed = sqliteTable("typed", { id: integer().primaryKey(), name: text().notNull() }) | ||
|
|
||
| // Same sweep as sqlite-interrupt.test.ts. It runs here because |
There was a problem hiding this comment.
This copied sweep is the most intricate part of a test that otherwise checks simple, deterministic results. It depends on MaxOpsBeforeYield and counted yieldNows, and runs 62 fresh client layers inside a subprocess. Your note in sqlite-interrupt.test.ts says the Node client shares makeSqliteAcquirers with the Bun client, which is already swept there, so it adds little coverage. I'd drop it (and leakedTransaction/leakedReserve from the expected output) to keep the Node test focused on WAL, errors and Drizzle queries.
Bun and Node now use Effect's own SQLite clients instead of ours
Before:
packages/corecarried its own SQLite client (database/sqlite.ts) with thin Bun and Node drivers on top. On Node every SQLite error came back asUnknownError, so a duplicate-key insert looked the same as a corrupt database. On Bun, opening withreadonly: truethrew, because we always passedcreate: true.After:
#sqlitestill resolves per runtime, but the Bun and Node entries are now short shims over@effect/sql-sqlite-bunand@effect/sql-sqlite-node4.1.0. Errors are classified on both (a duplicate key is aConstraintError), read-only opens work, and Node caches prepared statements.sqlite.tsis deleted. The workerd client stays ours for now; it no longer importssqlite.tsand keeps its scoped-transaction fix.What changes in behavior
sql.withTransactionnow opens withBEGIN IMMEDIATEinstead ofBEGIN. Direct callers: none in Bun or Node production code. The only one insrcisdrizzle/effect-sqlite/session.ts:141, on the workerd native path, which still uses our client. The rest are tests (sqlite-interrupt,sqlite-workerd).db.transactionis unchanged: it still reserves a connection and issues its ownbegin deferredand savepoints. Covered by the drizzle and new Node tests.busy_timeoutbefore the WAL pragma and retry WAL on a lock. The pragmas indatabase.ts(WAL, synchronous, busy_timeout, cache_size, wal_checkpoint, foreign_keys) still run after open.@effect/sql-sqlite-bunmoves from devDependencies to dependencies, and@effect/sql-sqlite-nodeis new (root catalog4.1.0).Bundles:
sqlite-bundle.test.tsnow checks that Node and default builds include@effect/sql-sqlite-nodebut neverbun:sqliteor@effect/sql-sqlite-bun, and that workerd includes neither upstream client. The Node CLI bundle (build-node.ts --bundle-only) builds, starts, and has nobun:sqlite.Tests: new
sqlite-node.test.tsbuilds a Node bundle and runs it undernode: WAL is on, a Drizzle transaction commits two rows, a duplicate insert fails withConstraintError, Drizzle's typedinsert().returning()andselect()work (including a constraint failure mid-returning()followed by a working query), and the queued-transaction and queued-reserve interrupt sweep fromsqlite-interrupt.test.tsleaks no connection. A Bun case checks the same error and reopens the file read-only. Both failed ondevwith the pinned Bun 1.4.4 (UnknownError, and Bun's "read-only database cannot have create: true"); older Bun versions accept that read-only open, so the Bun case only catches it on 1.4.4 and later.sqlite-interrupt.test.tspasses unchanged against the upstream Bun client.withTransaction5-row txwithTransaction5-row txBun is flat within noise; Node is faster, mostly from the statement cache. Not measured: cold reads on a large database, and lock contention with another process holding the file.
👀 Read the
BEGIN IMMEDIATEnote above andsqlite.workerd.ts; everything else is deletion or tests.Added by the opencode bot (AI-generated).
Related: #53879