Repository navigation
DataFusion reads Date32 and Date64 parquet statistics in as Int32Array #10587
Description
Describe the bug
When reading a Date32 or Date64 column from a parquet file, DataFusion currently returns an Int32 array
To Reproduce
You can see the issue in #10537
test_dates_32_diff_rg_sizestest_dates_64_diff_rg_sizes
Expected behavior
I expect a
- Date32 column to be read as
Date32Array - Date64 column to be read as
Date64Array
Additional context
No response
Activity
edmondop commentedon May 20, 2024
@alamb the title here doesn't make much sense, are you saying that the
minandmaxare not extracted as Date32/Date64?[-]DataFusion reads Date32 and Date64 parquet statistics in as [/-][+]DataFusion reads Date32 and Date64 parquet statistics in as Int32Array[/+]alamb commentedon May 20, 2024
Thanks for pointing that out @edmondop -- yes the min/max seem to be extracted as
Int32Arrays