You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Unsupported data type in hasher: Timestamp(Second, None) #1768
Describe the bug
Datafusion have 4 specific functions related to timestamp:
to_timestamp (nano)
to_timestamp_micros
to_timestamp_millis
to_timestamp_seconds
only to_timestamp_seconds is NOT supported in hasher, while other 3 are supported.
To Reproduce
use testing data
CREATE EXTERNAL TABLE x (
c9 BIGINT NOT NULL
)
STORED AS CSV
WITH HEADER ROW
LOCATION '/testing/data/csv/aggregate_test_100.csv';
to_timestamp_millis works in group by
select to_timestamp_millis(c9 * 1000), count(1) from x
group by 1
limit 1;
to_timestamp_seconds raise error in group by
select to_timestamp_seconds(c9), count(1) from x
group by 1
limit 1;
Internal error: Unsupported data type in hasher: Timestamp(Second, None). This was likely caused by a bug in DataFusion's code and we would welcome that you file an bug report in our issue tracker
Expected behavior
select to_timestamp_seconds(c9), count(1) from x
group by 1
limit 1;
Activity
jychen7 commentedon Feb 6, 2022
seems
TimeUnit::Secondis missing in following, will try to create a PR for ithttps://github.com/apache/arrow-datafusion/blob/af8786ebea1d62f2f4f28f3eb390cc73fd3a9e16/datafusion/src/physical_plan/hash_utils.rs#L390-L419
#1768 Support TimeUnit::Second in hasher (#1769)
Arrow2 02092022 (#1795)