You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
HUGEINT/UHUGEINT max-value overflows decimal128(38, 0) on RecordBatch return path #32
Round-tripping a HUGEINT or UHUGEINT column through the example.echo
table-in-out fixture (or any other path that re-serialises an Arrow batch
on the way back to DuckDB) raises:
vgi_rpc.utils.IPCError: IPC batch validation failed: In column 1: Invalid:
Decimal value 170141183460469231731687303715884105727 does not fit in
precision of decimal128(38, 0)
[worker: uv run --project vgi-fixture-worker]
Reproduction
Discovered while writing pushdown coverage in vgi/test/sql/integration/filter_pushdown/hugeint.test.
The HUGEINT max literal 170141183460469231731687303715884105727 (2^127 − 1, 39 digits) overflows the decimal128(38, 0) PyArrow type used on the echo
return path. Same issue with UHUGEINT max 340282366920938463463374607431768211455 (2^128 − 1, 39 digits).
The conversion happens on the worker side when echo's pass-through builds
a Record Batch from the input. HUGEINT is registered to round-trip through
the DuckDB.hugeint extension type
(vgi/duckdb/src/common/arrow/arrow_type_extension.cpp:558)
backed by fixed_size_binary(16), so the storage does fit; the decimal128(38, 0) only appears on echo's return-path serialisation. Look
at how vgi-rpc's IPC validator infers a target Arrow type for the columns
emitted by the worker — decimal128(38, 0) is one digit short of what
HUGEINT/UHUGEINT can hold.
Workaround in tests
The new hugeint.test exercises a 38-digit value
(99999999999999999999999999999999999999) instead of the type max. That
covers the kernel-pair fix without tripping the return-path bug.
Priority
Low — only affects the absolute boundary literals, very rare in real data.
But quietly silent-failing on max values is bad enough to deserve a fix or
a clear error.
Activity
table_filter_pushdown: handle canonical Arrow extension types