Skip to content

decodeBigIntMarkers / encodeBigIntMarkers turn binary values (ArrayBuffer, Uint8Array) into plain objects on the WebSocket transport #281

Description

@damlayildiz

Summary

On the WebSocket transport, binary values are destroyed by decodeBigIntMarkers and encodeBigIntMarkers in src/drivers/utilities.ts:

  • Reading: with safe_integer_mode: "mixed" (or "bigint"), a BLOB in a column that isn't declared BLOB comes back as an empty object {}.
  • Writing: a Uint8Array or ArrayBuffer bound as a query parameter is turned into a plain object before it is sent, in every mode.

Version: @sqlitecloud/drivers 1.0.878, browser, WebSocket connection.

Reproduce (reading)

const db = new Database({ /* websocket connection */, safe_integer_mode: "mixed" })
const rows = await db.sql`SELECT X'89504E47' AS png, randomblob(16) AS r, CAST('hi' AS BLOB) AS t`
console.log(rows[0]) // { png: {}, r: {}, t: {} }

Columns declared BLOB are unaffected: decodeWebsocketRowsetData decodes them from base64 into a Buffer before this point. Expression results, views, functions, and bytes stored in a column without a BLOB type are all affected.

Cause

decodeBigIntMarkers (src/drivers/utilities.ts, line 364) walks every cell. Any object that isn't a Node Buffer is rebuilt from Object.entries:

if (value && typeof value === 'object' && !Buffer.isBuffer(value)) {
  const result = {}
  Object.entries(value).forEach(([key, item]) => { result[key] = decodeBigIntMarkers(item, safeIntegerMode) })
  return result
}

In the browser those BLOB cells arrive as an ArrayBuffer, which has no own enumerable keys, so the result is {}.

encodeBigIntMarkers (line 343) has the same check. It runs on every bound parameter on the WebSocket transport (connection-ws.js: bind: encodeBigIntMarkers(commands.parameters)), regardless of safe_integer_mode.

Calling the published 1.0.878 functions directly gives:

Input Result
encodeBigIntMarkers(Buffer.from([1,2,3])) the Buffer, unchanged ✓
encodeBigIntMarkers(new Uint8Array([1,2,3])) {"0":1,"1":2,"2":3} (plain object) ✗
encodeBigIntMarkers(new Uint8Array([1,2,3]).buffer) {} ✗
decodeBigIntMarkers(new Uint8Array([1,2,3]).buffer, "mixed") {} ✗
decodeBigIntMarkers(new Uint8Array([1,2,3]), "mixed") {"0":1,"1":2,"2":3} ✗

(These are direct calls of the two functions, not a full round trip over a live connection. The reading case was also observed end to end in the browser.)

Suggested fix

Skip binary values in both functions, and ideally normalise an ArrayBuffer to a Buffer so every BLOB has one shape:

if (value instanceof ArrayBuffer) return Buffer.from(value)
if (ArrayBuffer.isView(value)) return value

Add tests showing that an untyped BLOB expression under mixed and bigint comes back as the original bytes, and that a Uint8Array or ArrayBuffer parameter is sent as bytes.

Impact

The SQLite Cloud dashboard's Studio connects over WebSocket with safe_integer_mode: "mixed". Any BLOB from an expression, a view, a function, or a column without a BLOB type shows as {} in the SQL Console and the data grid. Browser apps that bind a Uint8Array parameter would also send the wrong value.

No activity

Activity on this issue will appear here.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions