Skip to content

chore: bump datafusion to 49 - #4006

Merged
AdamGS merged 2 commits into
developfrom
adamg/df-49
Jul 28, 2025
Merged

chore: bump datafusion to 49#4006
AdamGS merged 2 commits into
developfrom
adamg/df-49

Conversation

@AdamGS

@AdamGS AdamGS commented Jul 24, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor

Waiting for apache/datafusion#16235 to merge.

Missing pieces here:

  • Figuring out how to wire vortex dynamic expressions and datafusion's new DynamicFilterPhysicalExpr

@AdamGS AdamGS added the changelog/chore A trivial change label Jul 24, 2025
@coveralls

coveralls commented Jul 24, 2025 •
edited
Loading

Copy link
Copy Markdown

Coverage Status

coverage: 81.78% (-0.009%) from 81.789%
when pulling b27b780 on adamg/df-49
into 3d13d2e on develop.

@github-actions

github-actions Bot commented Jul 24, 2025

Copy link
Copy Markdown
Contributor

Benchmarks: TPC-H on NVME

Table of Results
name PR cfc5971 base f25ddb6 ratio (PR/base) unit
tpch_q01/DataFusion:arrow 54556667 6.18381e+07 0.88225 ns
tpch_q02/DataFusion:arrow 17328534 1.66573e+07 1.0403 ns
tpch_q03/DataFusion:arrow 22195965 2.21418e+07 1.00245 ns
tpch_q04/DataFusion:arrow 12281135 1.21332e+07 1.01219 ns
tpch_q05/DataFusion:arrow 32942254 3.15246e+07 1.04497 ns
tpch_q06/DataFusion:arrow 7342140 7.09752e+06 1.03447 ns
tpch_q07/DataFusion:arrow 55111142 5.47925e+07 1.00581 ns
tpch_q08/DataFusion:arrow 19442419 1.7885e+07 1.08708 ns
tpch_q09/DataFusion:arrow 32588927 3.27041e+07 0.996477 ns
tpch_q10/DataFusion:arrow 29204388 2.90175e+07 1.00644 ns
tpch_q11/DataFusion:arrow 8344975 7.68303e+06 1.08616 ns
tpch_q12/DataFusion:arrow 21445539 2.37607e+07 0.902563 ns
tpch_q13/DataFusion:arrow 14509919 1.48256e+07 0.978704 ns
tpch_q14/DataFusion:arrow 7368433 6.95804e+06 1.05898 ns
tpch_q15/DataFusion:arrow 13760548 1.38869e+07 0.990904 ns
tpch_q16/DataFusion:arrow 15851207 1.49176e+07 1.06259 ns
tpch_q17/DataFusion:arrow 55387581 4.96707e+07 1.1151 ns
tpch_q18/DataFusion:arrow 116867718 1.10272e+08 1.05982 ns
tpch_q19/DataFusion:arrow 20144526 1.90845e+07 1.05555 ns
tpch_q20/DataFusion:arrow 22565772 2.04794e+07 1.10188 ns
tpch_q21/DataFusion:arrow 79604840 8.11572e+07 0.980872 ns
tpch_q22/DataFusion:arrow 10520303 1.04579e+07 1.00597 ns
tpch_q01/DataFusion:parquet 159852674 1.57471e+08 1.01513 ns
tpch_q02/DataFusion:parquet 83563973 8.05635e+07 1.03724 ns
tpch_q03/DataFusion:parquet 84023738 7.93669e+07 1.05867 ns
tpch_q04/DataFusion:parquet 64588897 5.71825e+07 1.12952 ns
tpch_q05/DataFusion:parquet 103872959 9.88515e+07 1.0508 ns
tpch_q06/DataFusion:parquet 35347716 3.61649e+07 0.977403 ns
tpch_q07/DataFusion:parquet 123149376 1.20425e+08 1.02262 ns
tpch_q08/DataFusion:parquet 97979998 9.42107e+07 1.04001 ns
tpch_q09/DataFusion:parquet 149550562 1.33962e+08 1.11637 ns
tpch_q10/DataFusion:parquet 132169079 1.25452e+08 1.05355 ns
tpch_q11/DataFusion:parquet 71230021 6.92523e+07 1.02856 ns
tpch_q12/DataFusion:parquet 88126130 8.65011e+07 1.01879 ns
tpch_q13/DataFusion:parquet 170763035 1.70228e+08 1.00314 ns
tpch_q14/DataFusion:parquet 44857013 4.30579e+07 1.04178 ns
tpch_q15/DataFusion:parquet 61369603 6.84296e+07 0.896829 ns
tpch_q16/DataFusion:parquet 50112385 4.76132e+07 1.05249 ns
tpch_q17/DataFusion:parquet 139769328 1.29329e+08 1.08073 ns
tpch_q18/DataFusion:parquet 187107457 1.7842e+08 1.04869 ns
tpch_q19/DataFusion:parquet 85356657 8.13414e+07 1.04936 ns
tpch_q20/DataFusion:parquet 73324956 6.87727e+07 1.06619 ns
tpch_q21/DataFusion:parquet 148280412 1.51951e+08 0.975846 ns
tpch_q22/DataFusion:parquet 77732200 7.40499e+07 1.04973 ns
tpch_q01/DataFusion:vortex-file-compressed 57711884 5.86479e+07 0.98404 ns
tpch_q02/DataFusion:vortex-file-compressed 33680958 3.07857e+07 1.09405 ns
tpch_q03/DataFusion:vortex-file-compressed 23047754 2.16223e+07 1.06593 ns
tpch_q04/DataFusion:vortex-file-compressed 16516218 1.63524e+07 1.01002 ns
tpch_q05/DataFusion:vortex-file-compressed 49909553 4.92339e+07 1.01372 ns
tpch_q06/DataFusion:vortex-file-compressed 8292334 7.7727e+06 1.06685 ns
tpch_q07/DataFusion:vortex-file-compressed 74011235 7.20649e+07 1.02701 ns
tpch_q08/DataFusion:vortex-file-compressed 39214783 3.64413e+07 1.07611 ns
tpch_q09/DataFusion:vortex-file-compressed 54138632 5.21388e+07 1.03836 ns
tpch_q10/DataFusion:vortex-file-compressed 42666377 4.16692e+07 1.02393 ns
tpch_q11/DataFusion:vortex-file-compressed 19966558 1.99647e+07 1.00009 ns
tpch_q12/DataFusion:vortex-file-compressed 17200286 1.66007e+07 1.03612 ns
tpch_q13/DataFusion:vortex-file-compressed 21484901 2.1523e+07 0.998231 ns
tpch_q14/DataFusion:vortex-file-compressed 11382497 1.07643e+07 1.05743 ns
tpch_q15/DataFusion:vortex-file-compressed 18733779 1.86628e+07 1.00381 ns
tpch_q16/DataFusion:vortex-file-compressed 24469998 2.3353e+07 1.04783 ns
tpch_q17/DataFusion:vortex-file-compressed 61407857 6.03226e+07 1.01799 ns
tpch_q18/DataFusion:vortex-file-compressed 95734644 9.5382e+07 1.0037 ns
tpch_q19/DataFusion:vortex-file-compressed 15927912 1.57812e+07 1.00929 ns
tpch_q20/DataFusion:vortex-file-compressed 24497570 2.38644e+07 1.02653 ns
tpch_q21/DataFusion:vortex-file-compressed 75114099 7.31139e+07 1.02736 ns
tpch_q22/DataFusion:vortex-file-compressed 15741078 1.55115e+07 1.0148 ns
tpch_q01/DuckDB:parquet 100926720 1.01185e+08 0.997452 ns
tpch_q02/DuckDB:parquet 44676086 4.44405e+07 1.0053 ns
tpch_q03/DuckDB:parquet 72768027 7.16595e+07 1.01547 ns
tpch_q04/DuckDB:parquet 57964694 5.66977e+07 1.02235 ns
tpch_q05/DuckDB:parquet 69494612 6.99166e+07 0.993964 ns
tpch_q06/DuckDB:parquet 20253823 2.00267e+07 1.01134 ns
tpch_q07/DuckDB:parquet 78789367 7.75756e+07 1.01565 ns
tpch_q08/DuckDB:parquet 90220569 9.05692e+07 0.996151 ns
tpch_q09/DuckDB:parquet 167812762 1.40182e+08 1.19711 ns
tpch_q10/DuckDB:parquet 127222142 1.29003e+08 0.986196 ns
tpch_q11/DuckDB:parquet 24118376 2.46004e+07 0.980407 ns
tpch_q12/DuckDB:parquet 58069572 5.83446e+07 0.995286 ns
tpch_q13/DuckDB:parquet 250353104 2.54274e+08 0.984579 ns
tpch_q14/DuckDB:parquet 46072081 4.56665e+07 1.00888 ns
tpch_q15/DuckDB:parquet 26941390 2.65448e+07 1.01494 ns
tpch_q16/DuckDB:parquet 47936287 4.80104e+07 0.998456 ns
tpch_q17/DuckDB:parquet 55456333 5.36782e+07 1.03313 ns
tpch_q18/DuckDB:parquet 124727158 1.23313e+08 1.01147 ns
tpch_q19/DuckDB:parquet 71052677 7.07068e+07 1.00489 ns
tpch_q20/DuckDB:parquet 59754873 6.03993e+07 0.989331 ns
tpch_q21/DuckDB:parquet 168308744 1.69016e+08 0.995815 ns
tpch_q22/DuckDB:parquet 51804491 5.05614e+07 1.02459 ns
tpch_q01/DuckDB:vortex-file-compressed 34265521 3.28292e+07 1.04375 ns
tpch_q02/DuckDB:vortex-file-compressed 25337180 2.58596e+07 0.979797 ns
tpch_q03/DuckDB:vortex-file-compressed 29165555 2.87473e+07 1.01455 ns
tpch_q04/DuckDB:vortex-file-compressed 29210997 2.71433e+07 1.07618 ns
tpch_q05/DuckDB:vortex-file-compressed 39233271 3.98839e+07 0.983687 ns
tpch_q06/DuckDB:vortex-file-compressed 8363978 7.9278e+06 1.05502 ns
tpch_q07/DuckDB:vortex-file-compressed 34758614 3.38552e+07 1.02669 ns
tpch_q08/DuckDB:vortex-file-compressed 45461948 4.36781e+07 1.04084 ns
tpch_q09/DuckDB:vortex-file-compressed 62261527 6.08683e+07 1.02289 ns
tpch_q10/DuckDB:vortex-file-compressed 42116312 4.01509e+07 1.04895 ns
tpch_q11/DuckDB:vortex-file-compressed 13838671 1.31349e+07 1.05358 ns
tpch_q12/DuckDB:vortex-file-compressed 27782471 2.70186e+07 1.02827 ns
tpch_q13/DuckDB:vortex-file-compressed 39520581 3.97524e+07 0.994169 ns
tpch_q14/DuckDB:vortex-file-compressed 19068605 1.86344e+07 1.0233 ns
tpch_q15/DuckDB:vortex-file-compressed 12702255 1.23419e+07 1.0292 ns
tpch_q16/DuckDB:vortex-file-compressed 23968906 2.35753e+07 1.0167 ns
tpch_q17/DuckDB:vortex-file-compressed 59576330 6.09882e+07 0.97685 ns
tpch_q18/DuckDB:vortex-file-compressed 50327102 5.02716e+07 1.0011 ns
tpch_q19/DuckDB:vortex-file-compressed 23203161 2.23602e+07 1.0377 ns
tpch_q20/DuckDB:vortex-file-compressed 30064784 2.90814e+07 1.03381 ns
tpch_q21/DuckDB:vortex-file-compressed 108447019 1.07809e+08 1.00592 ns
tpch_q22/DuckDB:vortex-file-compressed 22589180 2.24598e+07 1.00576 ns
tpch_q01/DuckDB:duckdb 14499384 1.43146e+07 1.01291 ns
tpch_q02/DuckDB:duckdb 11630179 1.1212e+07 1.0373 ns
tpch_q03/DuckDB:duckdb 15301547 1.52991e+07 1.00016 ns
tpch_q04/DuckDB:duckdb 17135249 1.67535e+07 1.02278 ns
tpch_q05/DuckDB:duckdb 16586928 1.6516e+07 1.0043 ns
tpch_q06/DuckDB:duckdb 3482660 3.43781e+06 1.01305 ns
tpch_q07/DuckDB:duckdb 19379718 1.90093e+07 1.01948 ns
tpch_q08/DuckDB:duckdb 18444539 1.84572e+07 0.999314 ns
tpch_q09/DuckDB:duckdb 54339190 5.49424e+07 0.989021 ns
tpch_q10/DuckDB:duckdb 45108529 4.03144e+07 1.11892 ns
tpch_q11/DuckDB:duckdb 7661996 6.62502e+06 1.15652 ns
tpch_q12/DuckDB:duckdb 13387894 1.2902e+07 1.03766 ns
tpch_q13/DuckDB:duckdb 47571922 4.52371e+07 1.05161 ns
tpch_q14/DuckDB:duckdb 12814886 1.29173e+07 0.992072 ns
tpch_q15/DuckDB:duckdb 7883363 7.83121e+06 1.00666 ns
tpch_q16/DuckDB:duckdb 21112140 2.06267e+07 1.02354 ns
tpch_q17/DuckDB:duckdb 14532733 1.45973e+07 0.995577 ns
tpch_q18/DuckDB:duckdb 35240423 3.49786e+07 1.00748 ns
tpch_q19/DuckDB:duckdb 22171865 2.23722e+07 0.991044 ns
tpch_q20/DuckDB:duckdb 17087557 1.66553e+07 1.02595 ns
tpch_q21/DuckDB:duckdb 52821872 5.19083e+07 1.0176 ns
tpch_q22/DuckDB:duckdb 24229998 2.32514e+07 1.04209 ns

@github-actions

github-actions Bot commented Jul 24, 2025

Copy link
Copy Markdown
Contributor

Benchmarks: TPC-H on S3

Table of Results
name PR cfc5971 base f25ddb6 ratio (PR/base) unit
tpch_q01/DataFusion:parquet 384963211 3.78358e+08 1.01746 ns
tpch_q02/DataFusion:parquet 764235088 7.56369e+08 1.0104 ns
tpch_q03/DataFusion:parquet 564874796 6.01331e+08 0.939373 ns
tpch_q04/DataFusion:parquet 299357206 3.05359e+08 0.980344 ns
tpch_q05/DataFusion:parquet 749226517 7.17953e+08 1.04356 ns
tpch_q06/DataFusion:parquet 271976201 2.58252e+08 1.05314 ns
tpch_q07/DataFusion:parquet 736914555 7.25719e+08 1.01543 ns
tpch_q08/DataFusion:parquet 804178538 8.3248e+08 0.966003 ns
tpch_q09/DataFusion:parquet 698025815 7.31985e+08 0.953607 ns
tpch_q10/DataFusion:parquet 738796696 7.45817e+08 0.990587 ns
tpch_q11/DataFusion:parquet 529939794 5.44916e+08 0.972517 ns
tpch_q12/DataFusion:parquet 385870737 3.90923e+08 0.987076 ns
tpch_q13/DataFusion:parquet 484647874 4.92071e+08 0.984914 ns
tpch_q14/DataFusion:parquet 333837963 3.38717e+08 0.985596 ns
tpch_q15/DataFusion:parquet 563140609 5.87848e+08 0.95797 ns
tpch_q16/DataFusion:parquet 366772861 3.48395e+08 1.05275 ns
tpch_q17/DataFusion:parquet 572014185 5.49517e+08 1.04094 ns
tpch_q18/DataFusion:parquet 682022317 6.80542e+08 1.00217 ns
tpch_q19/DataFusion:parquet 464639202 4.82667e+08 0.962649 ns
tpch_q20/DataFusion:parquet 517768991 5.37164e+08 0.963894 ns
tpch_q21/DataFusion:parquet 872266968 8.54401e+08 1.02091 ns
tpch_q22/DataFusion:parquet 352625088 3.22615e+08 1.09302 ns
tpch_q01/DataFusion:vortex-file-compressed 159395163 1.60792e+08 0.991311 ns
tpch_q02/DataFusion:vortex-file-compressed 134403726 1.51452e+08 0.887436 ns
tpch_q03/DataFusion:vortex-file-compressed 208240109 2.11963e+08 0.982437 ns
tpch_q04/DataFusion:vortex-file-compressed 134905635 1.40197e+08 0.962256 ns
tpch_q05/DataFusion:vortex-file-compressed 209447756 2.12065e+08 0.987658 ns
tpch_q06/DataFusion:vortex-file-compressed 95038848 9.38165e+07 1.01303 ns
tpch_q07/DataFusion:vortex-file-compressed 272178704 2.29422e+08 1.18637 ns
tpch_q08/DataFusion:vortex-file-compressed 272028872 2.4732e+08 1.0999 ns
tpch_q09/DataFusion:vortex-file-compressed 273133980 2.50034e+08 1.09239 ns
tpch_q10/DataFusion:vortex-file-compressed 245176884 2.7056e+08 0.906182 ns
tpch_q11/DataFusion:vortex-file-compressed 123820350 1.28717e+08 0.961959 ns
tpch_q12/DataFusion:vortex-file-compressed 181363267 1.67823e+08 1.08068 ns
tpch_q13/DataFusion:vortex-file-compressed 128158444 1.28526e+08 0.997143 ns
tpch_q14/DataFusion:vortex-file-compressed 144447550 1.43967e+08 1.00334 ns
tpch_q15/DataFusion:vortex-file-compressed 207656890 2.21602e+08 0.937071 ns
tpch_q16/DataFusion:vortex-file-compressed 85038257 8.37584e+07 1.01528 ns
tpch_q17/DataFusion:vortex-file-compressed 216657030 2.16605e+08 1.00024 ns
tpch_q18/DataFusion:vortex-file-compressed 278461999 2.58015e+08 1.07925 ns
tpch_q19/DataFusion:vortex-file-compressed 156801136 1.62222e+08 0.966585 ns
tpch_q20/DataFusion:vortex-file-compressed 195455244 2.20551e+08 0.886214 ns
tpch_q21/DataFusion:vortex-file-compressed 388689388 3.63112e+08 1.07044 ns
tpch_q22/DataFusion:vortex-file-compressed 102557406 1.05959e+08 0.967893 ns
tpch_q01/DuckDB:parquet 120481279 1.37982e+08 0.873167 ns
tpch_q02/DuckDB:parquet 392886129 3.29838e+08 1.19115 ns
tpch_q03/DuckDB:parquet 189992702 1.46591e+08 1.29607 ns
tpch_q04/DuckDB:parquet 112784519 9.9823e+07 1.12985 ns
tpch_q05/DuckDB:parquet 275828146 2.14423e+08 1.28638 ns
tpch_q06/DuckDB:parquet 34922545 6.45607e+07 0.540926 ns
tpch_q07/DuckDB:parquet 266444186 3.1115e+08 0.856321 ns
tpch_q08/DuckDB:parquet 412657110 3.90939e+08 1.05555 ns
tpch_q09/DuckDB:parquet 317021924 3.92558e+08 0.807581 ns
tpch_q10/DuckDB:parquet 242568322 2.65457e+08 0.913776 ns
tpch_q11/DuckDB:parquet 229294816 1.85913e+08 1.23334 ns
tpch_q12/DuckDB:parquet 127677103 1.18841e+08 1.07436 ns
tpch_q13/DuckDB:parquet 286981589 3.18872e+08 0.89999 ns
tpch_q14/DuckDB:parquet 98992848 1.27271e+08 0.777812 ns
tpch_q15/DuckDB:parquet 109581242 9.71539e+07 1.12791 ns
tpch_q16/DuckDB:parquet 136579762 1.46961e+08 0.929358 ns
tpch_q17/DuckDB:parquet 141383549 1.31023e+08 1.07908 ns
tpch_q18/DuckDB:parquet 243773465 2.20152e+08 1.1073 ns
tpch_q19/DuckDB:parquet 135119210 1.12712e+08 1.19881 ns
tpch_q20/DuckDB:parquet 204733273 2.26381e+08 0.904373 ns
tpch_q21/DuckDB:parquet 311538242 3.82776e+08 0.813891 ns
tpch_q22/DuckDB:parquet 146967648 1.41023e+08 1.04215 ns

@github-actions

github-actions Bot commented Jul 24, 2025

Copy link
Copy Markdown
Contributor

Benchmarks: Clickbench on NVME

Table of Results
name PR cfc5971 base f25ddb6 ratio (PR/base) unit
clickbench_q00/DataFusion:parquet 1587809 1.6551e+06 0.95934 ns
clickbench_q01/DataFusion:parquet 28485764 2.73763e+07 1.04053 ns
clickbench_q02/DataFusion:parquet 61450665 6.4574e+07 0.951632 ns
clickbench_q03/DataFusion:parquet 56297704 5.56951e+07 1.01082 ns
clickbench_q04/DataFusion:parquet 364513657 3.72472e+08 0.978633 ns
clickbench_q05/DataFusion:parquet 383612940 3.7906e+08 1.01201 ns
clickbench_q06/DataFusion:parquet 1622294 1.62564e+06 0.997939 ns
clickbench_q07/DataFusion:parquet 33448017 3.36585e+07 0.993747 ns
clickbench_q08/DataFusion:parquet 451078028 4.67782e+08 0.964291 ns
clickbench_q09/DataFusion:parquet 664279501 6.93446e+08 0.957939 ns
clickbench_q10/DataFusion:parquet 149305111 1.53073e+08 0.975382 ns
clickbench_q11/DataFusion:parquet 172299124 1.77644e+08 0.969914 ns
clickbench_q12/DataFusion:parquet 400447113 4.18334e+08 0.957242 ns
clickbench_q13/DataFusion:parquet 559945062 5.89931e+08 0.94917 ns
clickbench_q14/DataFusion:parquet 388949495 4.00133e+08 0.972051 ns
clickbench_q15/DataFusion:parquet 436738425 4.52931e+08 0.964249 ns
clickbench_q16/DataFusion:parquet 792313578 8.33604e+08 0.950468 ns
clickbench_q17/DataFusion:parquet 787447763 8.01373e+08 0.982623 ns
clickbench_q18/DataFusion:parquet 1486774244 1.53054e+09 0.971403 ns
clickbench_q19/DataFusion:parquet 43566092 4.68131e+07 0.930639 ns
clickbench_q20/DataFusion:parquet 659431818 6.47359e+08 1.01865 ns
clickbench_q21/DataFusion:parquet 738316241 7.32065e+08 1.00854 ns
clickbench_q22/DataFusion:parquet 1071930482 1.1014e+09 0.973242 ns
clickbench_q23/DataFusion:parquet 4754444495 4.893e+09 0.971684 ns
clickbench_q24/DataFusion:parquet 245129083 2.55021e+08 0.96121 ns
clickbench_q25/DataFusion:parquet 171469027 2.24221e+08 0.764731 ns
clickbench_q26/DataFusion:parquet 237220593 2.8824e+08 0.822997 ns
clickbench_q27/DataFusion:parquet 1084663949 1.09912e+09 0.986846 ns
clickbench_q28/DataFusion:parquet 5759261606 5.92746e+09 0.971625 ns
clickbench_q29/DataFusion:parquet 232971025 2.30646e+08 1.01008 ns
clickbench_q30/DataFusion:parquet 382644177 3.91922e+08 0.976327 ns
clickbench_q31/DataFusion:parquet 422505113 4.48629e+08 0.941769 ns
clickbench_q32/DataFusion:parquet 1356055976 1.37052e+09 0.989443 ns
clickbench_q33/DataFusion:parquet 1680530738 1.6967e+09 0.990469 ns
clickbench_q34/DataFusion:parquet 1738345058 1.73206e+09 1.00363 ns
clickbench_q35/DataFusion:parquet 619405758 6.18052e+08 1.00219 ns
clickbench_q36/DataFusion:parquet 178269149 1.90303e+08 0.936766 ns
clickbench_q37/DataFusion:parquet 65972692 7.07204e+07 0.932866 ns
clickbench_q38/DataFusion:parquet 100226700 1.06379e+08 0.942164 ns
clickbench_q39/DataFusion:parquet 318366878 3.40421e+08 0.935216 ns
clickbench_q40/DataFusion:parquet 40471110 4.30546e+07 0.939995 ns
clickbench_q41/DataFusion:parquet 34968442 3.92943e+07 0.88991 ns
clickbench_q42/DataFusion:parquet 50694342 5.15504e+07 0.983393 ns
clickbench_q00/DataFusion:vortex-file-compressed 1772692 1.74638e+06 1.01507 ns
clickbench_q01/DataFusion:vortex-file-compressed 8701266 9.63573e+06 0.903021 ns
clickbench_q02/DataFusion:vortex-file-compressed 27140136 2.88893e+07 0.939454 ns
clickbench_q03/DataFusion:vortex-file-compressed 36920620 3.70404e+07 0.996765 ns
clickbench_q04/DataFusion:vortex-file-compressed 322415394 3.19176e+08 1.01015 ns
clickbench_q05/DataFusion:vortex-file-compressed 340550401 3.6258e+08 0.939242 ns
clickbench_q06/DataFusion:vortex-file-compressed 1679887 1.70782e+06 0.983642 ns
clickbench_q07/DataFusion:vortex-file-compressed 12561238 1.29037e+07 0.973459 ns
clickbench_q08/DataFusion:vortex-file-compressed 394583695 3.90675e+08 1.01 ns
clickbench_q09/DataFusion:vortex-file-compressed 574205265 5.80852e+08 0.988557 ns
clickbench_q10/DataFusion:vortex-file-compressed 87296046 8.88929e+07 0.982036 ns
clickbench_q11/DataFusion:vortex-file-compressed 98504347 9.99668e+07 0.98537 ns
clickbench_q12/DataFusion:vortex-file-compressed 300147125 3.11039e+08 0.964981 ns
clickbench_q13/DataFusion:vortex-file-compressed 477469036 5.03445e+08 0.948404 ns
clickbench_q14/DataFusion:vortex-file-compressed 277019301 2.91182e+08 0.951363 ns
clickbench_q15/DataFusion:vortex-file-compressed 370770863 3.92418e+08 0.944836 ns
clickbench_q16/DataFusion:vortex-file-compressed 788834007 8.35185e+08 0.944502 ns
clickbench_q17/DataFusion:vortex-file-compressed 795905833 8.02471e+08 0.991819 ns
clickbench_q18/DataFusion:vortex-file-compressed 1425890691 1.54204e+09 0.924676 ns
clickbench_q19/DataFusion:vortex-file-compressed 18673237 1.7619e+07 1.05983 ns
clickbench_q20/DataFusion:vortex-file-compressed 378460422 3.32524e+08 1.13814 ns
clickbench_q21/DataFusion:vortex-file-compressed 389669525 4.11036e+08 0.948017 ns
clickbench_q22/DataFusion:vortex-file-compressed 786719029 8.25675e+08 0.952819 ns
clickbench_q23/DataFusion:vortex-file-compressed 2922742928 3.0963e+09 0.943948 ns
clickbench_q24/DataFusion:vortex-file-compressed 133089404 1.51181e+08 0.880334 ns
clickbench_q25/DataFusion:vortex-file-compressed 76829365 1.25039e+08 0.614443 ns
clickbench_q26/DataFusion:vortex-file-compressed 128524801 1.83356e+08 0.700957 ns
clickbench_q27/DataFusion:vortex-file-compressed 829673816 8.41245e+08 0.986246 ns
clickbench_q28/DataFusion:vortex-file-compressed 6648296909 6.71961e+09 0.989388 ns
clickbench_q29/DataFusion:vortex-file-compressed 224544961 2.26166e+08 0.99283 ns
clickbench_q30/DataFusion:vortex-file-compressed 261381220 2.76531e+08 0.945216 ns
clickbench_q31/DataFusion:vortex-file-compressed 298521804 3.18999e+08 0.935807 ns
clickbench_q32/DataFusion:vortex-file-compressed 1310791703 1.42581e+09 0.919328 ns
clickbench_q33/DataFusion:vortex-file-compressed 1537986431 1.6769e+09 0.91716 ns
clickbench_q34/DataFusion:vortex-file-compressed 1537246136 1.59619e+09 0.963073 ns
clickbench_q35/DataFusion:vortex-file-compressed 556807500 5.4497e+08 1.02172 ns
clickbench_q36/DataFusion:vortex-file-compressed 112031130 1.0989e+08 1.01949 ns
clickbench_q37/DataFusion:vortex-file-compressed 51150209 4.86424e+07 1.05156 ns
clickbench_q38/DataFusion:vortex-file-compressed 36801567 3.61908e+07 1.01688 ns
clickbench_q39/DataFusion:vortex-file-compressed 220110263 2.05184e+08 1.07275 ns
clickbench_q40/DataFusion:vortex-file-compressed 20067520 2.23352e+07 0.898469 ns
clickbench_q41/DataFusion:vortex-file-compressed 20136609 1.92577e+07 1.04564 ns
clickbench_q42/DataFusion:vortex-file-compressed 34013013 3.42627e+07 0.992714 ns
clickbench_q00/DuckDB:parquet 131312399 1.3053e+08 1.00599 ns
clickbench_q01/DuckDB:parquet 54971256 5.54899e+07 0.990653 ns
clickbench_q02/DuckDB:parquet 70560392 7.02134e+07 1.00494 ns
clickbench_q03/DuckDB:parquet 67952424 7.04421e+07 0.964657 ns
clickbench_q04/DuckDB:parquet 218389130 2.14707e+08 1.01715 ns
clickbench_q05/DuckDB:parquet 241400692 2.42654e+08 0.994837 ns
clickbench_q06/DuckDB:parquet 75014609 7.34851e+07 1.02081 ns
clickbench_q07/DuckDB:parquet 64837115 6.68387e+07 0.970054 ns
clickbench_q08/DuckDB:parquet 282930579 2.84612e+08 0.994092 ns
clickbench_q09/DuckDB:parquet 413333193 4.12905e+08 1.00104 ns
clickbench_q10/DuckDB:parquet 110410601 1.11107e+08 0.993736 ns
clickbench_q11/DuckDB:parquet 120563839 1.23803e+08 0.973838 ns
clickbench_q12/DuckDB:parquet 253443662 2.58162e+08 0.981722 ns
clickbench_q13/DuckDB:parquet 422227095 4.21182e+08 1.00248 ns
clickbench_q14/DuckDB:parquet 287903604 2.90329e+08 0.991645 ns
clickbench_q15/DuckDB:parquet 245924859 2.46135e+08 0.999145 ns
clickbench_q16/DuckDB:parquet 585723799 5.8676e+08 0.998235 ns
clickbench_q17/DuckDB:parquet 503536254 5.05405e+08 0.996302 ns
clickbench_q18/DuckDB:parquet 972231470 9.83143e+08 0.988901 ns
clickbench_q19/DuckDB:parquet 65898735 6.76894e+07 0.973545 ns
clickbench_q20/DuckDB:parquet 395142377 3.82157e+08 1.03398 ns
clickbench_q21/DuckDB:parquet 502578078 5.00328e+08 1.0045 ns
clickbench_q22/DuckDB:parquet 875945772 8.76454e+08 0.99942 ns
clickbench_q23/DuckDB:parquet 277550223 2.87191e+08 0.96643 ns
clickbench_q24/DuckDB:parquet 94741175 1.12074e+08 0.845347 ns
clickbench_q25/DuckDB:parquet 142148777 1.41242e+08 1.00642 ns
clickbench_q26/DuckDB:parquet 62692722 6.54774e+07 0.957471 ns
clickbench_q27/DuckDB:parquet 637084801 6.42519e+08 0.991543 ns
clickbench_q28/DuckDB:parquet 5709128596 5.7023e+09 1.0012 ns
clickbench_q29/DuckDB:parquet 74933235 7.59137e+07 0.987084 ns
clickbench_q30/DuckDB:parquet 295969589 2.99167e+08 0.989313 ns
clickbench_q31/DuckDB:parquet 341758652 3.46511e+08 0.986286 ns
clickbench_q32/DuckDB:parquet 1030967665 1.03595e+09 0.995192 ns
clickbench_q33/DuckDB:parquet 1069489976 1.06474e+09 1.00446 ns
clickbench_q34/DuckDB:parquet 1125193752 1.13493e+09 0.991419 ns
clickbench_q35/DuckDB:parquet 376799707 3.79059e+08 0.99404 ns
clickbench_q36/DuckDB:parquet 72289795 6.76223e+07 1.06902 ns
clickbench_q37/DuckDB:parquet 63611270 6.30698e+07 1.00859 ns
clickbench_q38/DuckDB:parquet 68776057 7.37473e+07 0.932591 ns
clickbench_q39/DuckDB:parquet 101375864 1.05286e+08 0.962864 ns
clickbench_q40/DuckDB:parquet 65000022 6.85122e+07 0.948736 ns
clickbench_q41/DuckDB:parquet 70971795 7.76083e+07 0.914487 ns
clickbench_q42/DuckDB:parquet 56399039 5.86283e+07 0.961976 ns
clickbench_q00/DuckDB:vortex-file-compressed 9870941 1.04878e+07 0.941187 ns
clickbench_q01/DuckDB:vortex-file-compressed 12679570 1.40072e+07 0.905221 ns
clickbench_q02/DuckDB:vortex-file-compressed 28420308 2.78403e+07 1.02083 ns
clickbench_q03/DuckDB:vortex-file-compressed 34334595 3.37059e+07 1.01865 ns
clickbench_q04/DuckDB:vortex-file-compressed 212952908 2.13526e+08 0.997315 ns
clickbench_q05/DuckDB:vortex-file-compressed 160486635 1.59851e+08 1.00398 ns
clickbench_q06/DuckDB:vortex-file-compressed 38379503 3.82651e+07 1.00299 ns
clickbench_q07/DuckDB:vortex-file-compressed 20085221 2.02422e+07 0.992247 ns
clickbench_q08/DuckDB:vortex-file-compressed 279422870 2.7877e+08 1.00234 ns
clickbench_q09/DuckDB:vortex-file-compressed 396578514 3.96891e+08 0.999211 ns
clickbench_q10/DuckDB:vortex-file-compressed 89071718 9.03484e+07 0.98587 ns
clickbench_q11/DuckDB:vortex-file-compressed 99118877 9.9956e+07 0.991625 ns
clickbench_q12/DuckDB:vortex-file-compressed 183813992 1.83481e+08 1.00182 ns
clickbench_q13/DuckDB:vortex-file-compressed 365944859 3.68471e+08 0.993145 ns
clickbench_q14/DuckDB:vortex-file-compressed 207711550 2.08212e+08 0.997598 ns
clickbench_q15/DuckDB:vortex-file-compressed 236975522 2.38535e+08 0.993461 ns
clickbench_q16/DuckDB:vortex-file-compressed 518355984 5.21897e+08 0.993215 ns
clickbench_q17/DuckDB:vortex-file-compressed 435831331 4.4203e+08 0.985978 ns
clickbench_q18/DuckDB:vortex-file-compressed 895549749 9.07618e+08 0.986704 ns
clickbench_q19/DuckDB:vortex-file-compressed 222998790 2.26594e+08 0.984132 ns
clickbench_q20/DuckDB:vortex-file-compressed 291309662 2.92162e+08 0.997082 ns
clickbench_q21/DuckDB:vortex-file-compressed 362324668 3.72226e+08 0.973398 ns
clickbench_q22/DuckDB:vortex-file-compressed 579183171 5.85494e+08 0.989222 ns
clickbench_q23/DuckDB:vortex-file-compressed 290801942 2.9806e+08 0.975648 ns
clickbench_q24/DuckDB:vortex-file-compressed 24175720 2.40771e+07 1.00409 ns
clickbench_q25/DuckDB:vortex-file-compressed 85235514 8.46858e+07 1.00649 ns
clickbench_q26/DuckDB:vortex-file-compressed 23947851 2.4057e+07 0.995461 ns
clickbench_q27/DuckDB:vortex-file-compressed 548857111 5.4905e+08 0.999648 ns
clickbench_q28/DuckDB:vortex-file-compressed 5591575063 5.58969e+09 1.00034 ns
clickbench_q29/DuckDB:vortex-file-compressed 33002666 3.23773e+07 1.01932 ns
clickbench_q30/DuckDB:vortex-file-compressed 193697761 1.9341e+08 1.00149 ns
clickbench_q31/DuckDB:vortex-file-compressed 254565488 2.58782e+08 0.983707 ns
clickbench_q32/DuckDB:vortex-file-compressed 1031336313 1.05872e+09 0.974131 ns
clickbench_q33/DuckDB:vortex-file-compressed 926739729 9.36765e+08 0.989298 ns
clickbench_q34/DuckDB:vortex-file-compressed 1052611132 1.04903e+09 1.00342 ns
clickbench_q35/DuckDB:vortex-file-compressed 387790559 3.86761e+08 1.00266 ns
clickbench_q36/DuckDB:vortex-file-compressed 22059151 2.07277e+07 1.06424 ns
clickbench_q37/DuckDB:vortex-file-compressed 15160213 1.49728e+07 1.01251 ns
clickbench_q38/DuckDB:vortex-file-compressed 16253688 1.56674e+07 1.03742 ns
clickbench_q39/DuckDB:vortex-file-compressed 36489564 3.6148e+07 1.00945 ns
clickbench_q40/DuckDB:vortex-file-compressed 13150657 1.22764e+07 1.07122 ns
clickbench_q41/DuckDB:vortex-file-compressed 14314779 1.33574e+07 1.07168 ns
clickbench_q42/DuckDB:vortex-file-compressed 16916762 1.61804e+07 1.04551 ns
clickbench_q00/DuckDB:duckdb 16404585 1.71588e+07 0.956048 ns
clickbench_q01/DuckDB:duckdb 5702928 5.63949e+06 1.01125 ns
clickbench_q02/DuckDB:duckdb 15346159 1.53975e+07 0.996664 ns
clickbench_q03/DuckDB:duckdb 19446768 1.9484e+07 0.998089 ns
clickbench_q04/DuckDB:duckdb 203695580 1.9824e+08 1.02752 ns
clickbench_q05/DuckDB:duckdb 242074461 2.4292e+08 0.99652 ns
clickbench_q06/DuckDB:duckdb 14461527 1.49437e+07 0.967737 ns
clickbench_q07/DuckDB:duckdb 5769226 5.95395e+06 0.968974 ns
clickbench_q08/DuckDB:duckdb 253395714 2.53428e+08 0.999872 ns
clickbench_q09/DuckDB:duckdb 359291052 3.5818e+08 1.0031 ns
clickbench_q10/DuckDB:duckdb 77078795 7.75243e+07 0.994253 ns
clickbench_q11/DuckDB:duckdb 83962114 8.41477e+07 0.997795 ns
clickbench_q12/DuckDB:duckdb 206489835 2.0882e+08 0.988842 ns
clickbench_q13/DuckDB:duckdb 348741127 3.50902e+08 0.993843 ns
clickbench_q14/DuckDB:duckdb 218709957 2.19503e+08 0.996389 ns
clickbench_q15/DuckDB:duckdb 228082629 2.2645e+08 1.00721 ns
clickbench_q16/DuckDB:duckdb 507746670 5.09749e+08 0.996071 ns
clickbench_q17/DuckDB:duckdb 429620648 4.30677e+08 0.997547 ns
clickbench_q18/DuckDB:duckdb 857686948 8.50456e+08 1.0085 ns
clickbench_q19/DuckDB:duckdb 2523040 2.51465e+06 1.00334 ns
clickbench_q20/DuckDB:duckdb 295499906 2.95354e+08 1.00049 ns
clickbench_q21/DuckDB:duckdb 252247292 2.48612e+08 1.01462 ns
clickbench_q22/DuckDB:duckdb 390365121 3.85854e+08 1.01169 ns
clickbench_q23/DuckDB:duckdb 45205066 4.40188e+07 1.02695 ns
clickbench_q24/DuckDB:duckdb 15259466 1.55994e+07 0.978209 ns
clickbench_q25/DuckDB:duckdb 78832013 7.89549e+07 0.998444 ns
clickbench_q26/DuckDB:duckdb 14892128 1.39458e+07 1.06786 ns
clickbench_q27/DuckDB:duckdb 381542209 3.84677e+08 0.991851 ns
clickbench_q28/DuckDB:duckdb 5600984456 5.5308e+09 1.01269 ns
clickbench_q29/DuckDB:duckdb 21813791 2.21173e+07 0.986278 ns
clickbench_q30/DuckDB:duckdb 195765550 1.97328e+08 0.992082 ns
clickbench_q31/DuckDB:duckdb 238674191 2.40254e+08 0.993423 ns
clickbench_q32/DuckDB:duckdb 990649232 9.72631e+08 1.01853 ns
clickbench_q33/DuckDB:duckdb 874442449 8.68762e+08 1.00654 ns
clickbench_q34/DuckDB:duckdb 945644363 9.49688e+08 0.995742 ns
clickbench_q35/DuckDB:duckdb 300811342 3.09514e+08 0.971884 ns
clickbench_q36/DuckDB:duckdb 26379518 2.49137e+07 1.05884 ns
clickbench_q37/DuckDB:duckdb 5192010 5.97518e+06 0.86893 ns
clickbench_q38/DuckDB:duckdb 7413592 9.82692e+06 0.754416 ns
clickbench_q39/DuckDB:duckdb 45637229 4.5495e+07 1.00313 ns
clickbench_q40/DuckDB:duckdb 5669029 4.2834e+06 1.32349 ns
clickbench_q41/DuckDB:duckdb 5426301 5.22009e+06 1.0395 ns
clickbench_q42/DuckDB:duckdb 7439898 8.50722e+06 0.874539 ns

@robert3005

robert3005 commented Jul 24, 2025

Copy link
Copy Markdown
Contributor
  • Add conversion of DynamicFilterPhysicalExpr into our dynamic expressions

@AdamGS
AdamGS force-pushed the adamg/df-49 branch 3 times, most recently from a5a0c4d to bb9e133 Compare July 28, 2025 09:36
AdamGS added 2 commits July 28, 2025 10:45
@AdamGS
chore: bump datafusion to 49
035df82 
Signed-off-by: Adam Gutglick <adam@spiraldb.com>
@AdamGS
lint and bump deps
b27b780 
Signed-off-by: Adam Gutglick <adam@spiraldb.com>
@AdamGS
AdamGS marked this pull request as ready for review July 28, 2025 09:46
@AdamGS
AdamGS requested a review from robert3005 July 28, 2025 09:46
@AdamGS
AdamGS enabled auto-merge (squash) July 28, 2025 09:46
robert3005
robert3005 approved these changes Jul 28, 2025
View reviewed changes

@robert3005 robert3005 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should see if we care about any of the changes to the FileOpener

@AdamGS
AdamGS merged commit a0efcbe into develop Jul 28, 2025
@AdamGS
AdamGS deleted the adamg/df-49 branch July 28, 2025 09:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@robert3005 robert3005 robert3005 approved these changes

Assignees

No one assigned

Labels

changelog/chore A trivial change

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@AdamGS @coveralls @robert3005