feat: add ASOF join physical operator by Xuanwo · Pull Request #23828 · apache/datafusion

Xuanwo · 2026-07-23T05:06:03Z

Which issue does this PR close?

Part of ASOF join support / Specialize Range Joins #318.
Umbrella PR: feat: support ASOF joins #23738.
Follow-up for floating-point equality keys: feat: support floating-point ASOF equality keys #24375.

Rationale for this change

This is the first layer of the ASOF JOIN stack. It establishes a
broadcast-based physical execution contract independently so later
floating-point equality, logical-plan, SQL, DataFrame, and serialization changes
can be reviewed as smaller follow-up PRs.

The initial implementation deliberately favors the simpler broadcast design:
the right input must fit in memory and each left partition scans the shared
right-side batches. A repartitioned implementation can be evaluated separately
without changing the ASOF semantics introduced here.

Floating-point equality keys are rejected in this base layer because Arrow's
required sort order distinguishes -0.0 from +0.0 while join equality does
not. #24375 adds the required ordering normalization as an independently
reviewable layer.

What changes are included in this PR?

Add AsOfJoinExec for left-preserving, Snowflake-style ASOF semantics.
Coalesce and collect the ordered right input once, then share it across all
left partitions.
Keep the left input partitioned so each partition can scan independently and
preserve the left-side output partitioning.
Preserve merge state across input and output batch boundaries.
Reserve each retained Arrow buffer exactly once, including when right-side
batches are zero-copy slices, and expose build, match, and output metrics.
Define output properties and statistics for the broadcast execution model.
Reject floating-point equality keys until feat: support floating-point ASOF equality keys #24375 supplies a sort/equality
contract that handles signed zero correctly.
Add physical operator tests covering match directions, equality groups,
batch boundaries, unmatched rows, invalid contracts, shared-buffer memory
accounting, multi-partition broadcast execution, and float-key rejection.

Are these changes tested?

Yes:

cargo fmt --all
cargo clippy --all-targets --all-features -- -D warnings
cargo test -p datafusion-physical-plan joins::asof_join --all-features
Extended workspace tests from the contributor guide
FFI integration tests

Are there any user-facing changes?

This adds a new physical operator API. The base operator deliberately rejects
floating-point equality keys; #24375 adds full Float16, Float32, and Float64
support. SQL and DataFrame APIs are left to later dependent PRs.

codecov-commenter · 2026-07-23T05:34:34Z

Codecov Report

❌ Patch coverage is 79.10818% with 253 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.21%. Comparing base (ec110ce) to head (5c43ae3).
⚠️ Report is 42 commits behind head on main.

Files with missing lines	Patch %	Lines
datafusion/physical-plan/src/joins/asof_join.rs	79.10%	197 Missing and 56 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #23828      +/-   ##
==========================================
+ Coverage   81.19%   81.21%   +0.02%     
==========================================
  Files        1110     1114       +4     
  Lines      388618   393461    +4843     
  Branches   388618   393461    +4843     
==========================================
+ Hits       315531   319542    +4011     
- Misses      54507    55113     +606     
- Partials    18580    18806     +226

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

2010YOUY01

Thank you! This is a really good start. I have done a quick first pass and left some suggestions.

2010YOUY01 · 2026-07-31T08:55:55Z

+        vec![ChildStats::At(partition), ChildStats::Skip]
+    }
+
+    fn statistics_from_inputs(


Just an idea to make this PR smaller, could we use the default implementation here? We could implement it later in a follow-up PR.

I kept this small override because ASOF has two exact facts the default would discard: the output row count equals the left row count, and unmodified left columns retain their statistics. Right-side column statistics remain unknown. I added a comment to make that scope explicit.

I see, this makes sense.

Thanks! I also updated it to preserve the same contract through the general projection.

2010YOUY01 · 2026-08-06T08:14:09Z

There is no new commits after the previous review, you may have forgotten to push the local changes 🤔 @Xuanwo

Xuanwo · 2026-08-07T06:25:03Z

There is no new commits after the previous review, you may have forgotten to push the local changes 🤔 @Xuanwo

Oh, sorry, latest commit is a127baa

2010YOUY01

I went over the implementation in detail, and I think it's well organized.

Need to do before merging

Before merging, I think we could add a few basic tests that run the executor and assert the results, just to cover some different cases:

No equality condition in on.
Different comparison operators in the match condition, such as < and >=.
Complex expressions in the on or match conditions, such as MATCH_CONDITION (l_c1 + l_c2) > r_c1 or ON l_c1 = (r_c1 + r_c2). I think these should be supported.

Optional suggestions

The main implementation/design questions I have are:

We need to buffer batch_size output rows before emitting them, now it's implemented through the PendingRows struct, so we need to buffer all source batches to produce the final output. I think this uses more memory and makes the implementation more complex. An alternative is a) keep a in_progress_batch, and emit it after it reaches threshould b) only buffer one (left_batch, right_batch), and materialize its valid indices into the in_progress_batch when the cursor moves across it. This might be fast enough and simpler.
Each loop iteration only advances the right index by one. We can probably explore some fast-forwarding optimization here.

But I suggest we first implement this end to end, including planning, SQL support, more tests, and benchmarks, before exploring these optimizations further. It should be easier to validate the ideas afterward.

For now, I only suggest trying to simplify the existing implementation or adding more documentation to make future iterations easier. I left a few suggestions in the comments.

2010YOUY01 · 2026-08-10T08:24:52Z

+
+    fn input_distribution_requirements(&self) -> InputDistributionRequirements {
+        InputDistributionRequirements::new(vec![
+            Distribution::UnspecifiedDistribution,


Is it the case the optimizer will insert round-robin repartition, if we declare this UnspecifiedDistribution? It should be fine if it's doing so, I was a little bit confused by this name 🤔

2010YOUY01 · 2026-08-10T08:25:30Z

+        vec![ChildStats::At(partition), ChildStats::Skip]
+    }
+
+    fn statistics_from_inputs(


I see, this makes sense.

Co-authored-by: Yongting You <2010youy01@gmail.com>

Xuanwo · 2026-08-11T11:20:59Z

Thanks for the detailed review — this was very helpful. I added result-based coverage for no equality keys, both >= and <, and complex equality/match expressions. I also aligned projection with the existing join convention, simplified the metrics and tests, and documented the state-machine loop. I left the PendingRows redesign and right-cursor fast-forwarding for a benchmark-backed follow-up after the end-to-end SQL integration, as suggested.

Xuanwo · 2026-08-14T15:36:55Z

I'm not so sure if there is an easy fix, it it's hard we could first reject floats in the planning, and fix that in a separate PR.

Checked a bit, seems to be a mid level fix. Let's fix that in a seperate PR #24375. We can get this in first! 💌

jayzhan211

I hope we could have ASOFJoinStream that handles the AsOfJoinStreamState and performs the actual join. We could take inspiration from the existing join implementations, such as HashJoinStream.

jayzhan211 · 2026-08-16T13:11:30Z

+#[derive(Default)]
+struct PendingRows {
+    /// Distinct source batches referenced by `indices`.
+    sources: Vec<Arc<RecordBatch>>,


Do we need Vec<Arc<RecordBatch>> and not just Vec<RecordBatch>

Yep, Arc is intentional here: it keeps per-row clones O(1) and gives PendingRows a stable identity for deduplication. Added a short comment.

2010YOUY01

Thanks again!

2010YOUY01 · 2026-08-17T04:13:35Z

    /// flush pending rows without resetting either cursor or the candidate
    /// ```
-    async fn next_batch(&mut self) -> Result<Option<RecordBatch>> {
+    fn poll_next_impl(


I haven't fully figure out what's the best practice to implement the state machines to make them easy to read and extend, probably it is:

generator pattern [Status Update] Simplifing Streams to be more textbook-like and have less state while keeping the same perf #23974

If we need to draw a state machine to understand its mechanism, then use explicit state management like

datafusion/datafusion/physical-plan/src/aggregates/hash_stream.rs

Line 175 in 9a83691

enum PartialHashAggregateState {

Not an issue for now, we could experiment in the future.

Xuanwo · 2026-08-17T06:39:31Z

cc @jayzhan211, @xudong963, and @alamb, would you like to take another look and move forward?

2010YOUY01 · 2026-08-17T08:29:52Z

cc @jayzhan211, @xudong963, and @alamb, would you like to take another look and move forward?

datafusion project typically wait for 24 hrs after approval to merge, so others can have a chance to look. This one is a larger PR, probably we could wait for 2 days, unless anyone need more time to review.

https://datafusion.apache.org/contributor-guide/pr_review.html#pr-review-mechanics

Xuanwo · 2026-08-17T14:01:25Z

Got it, thank you @2010YOUY01 for the explanation 🙌

jayzhan211

I think we could try to rewrite it with more straightforward way in the follow up PR 🤔

jayzhan211 · 2026-08-18T13:12:22Z

+        ]
+    }
+
+    fn maintains_input_order(&self) -> Vec<bool> {


Suggested change

fn maintains_input_order(&self) -> Vec<bool> {

// ASOF emits exactly one row for each left row and never reorders the

// left input. The right input is scanned independently.

vec![true, false]

alamb · 2026-08-18T19:54:49Z

cc @jayzhan211, @xudong963, and @alamb, would you like to take another look and move forward?

datafusion project typically wait for 24 hrs after approval to merge, so others can have a chance to look. This one is a larger PR, probably we could wait for 2 days, unless anyone need more time to review.

https://datafusion.apache.org/contributor-guide/pr_review.html#pr-review-mechanics

Given @2010YOUY01 and @jayzhan211 have reviewed and approved I am happy to have this merged without my review (I am currently focusing a bit lower with arrow/parquet and trying to keep them going along)

I did however add this feature to the list of things to list in the next release when we get there

Release DataFusion 56.0.0 (Oct/Nov 2026) #24461

Nice work

Xuanwo · 2026-08-20T14:31:53Z

Thank you @2010YOUY01 again for getting this PR merged! Do you want to review the whole stack? Can I ping you on the PR while I'm ready?

2010YOUY01 · 2026-08-20T15:00:10Z

Sure, I'm interested in helping with this project 🫡

alamb · 2026-08-24T21:24:31Z

## Which issue does this PR close? - Part of #318. - Umbrella PR: #23738. - Depends on #23828 (merged). ## Rationale for this change This is the logical-planning layer of the ASOF JOIN stack. It defines the logical contract and planner behavior separately from the SQL frontend and serialization formats. #23828 is merged, so this PR's diff against `main` is the isolated logical layer. It no longer depends on the optional floating-point follow-up #24375. ## What changes are included in this PR? - Add `LogicalPlan::AsOfJoin`, `AsOfJoin`, and `AsOfMatch`. - Validate deterministic expressions, input ownership, supported match operators, equality-key types, and USING constraints. - Add `LogicalPlanBuilder` entry points and schema construction that preserves both qualified `USING` keys while exposing one unqualified wildcard key. - Integrate ASOF joins with tree transforms, display, type coercion, projection pruning, row bounds, and physical planning. - Plan logical ASOF joins to the broadcast-based `AsOfJoinExec` from #23828. - Fail closed at proto, SQL unparser, and Substrait boundaries until their owning stack layers add explicit support. - Defer ASOF-specific functional-dependency refinement to #24799 and filter pushdown to #24801 so each optimization can be reviewed independently. ## Are these changes tested? Yes: - `cargo fmt --all` - `cargo clippy --all-targets --all-features -- -D warnings` - `cargo test -p datafusion-expr min_rows_of_joins --all-features` - `cargo test -p datafusion-substrait asof_join_fails_closed_until_substrait_has_an_extension --all-features` - The extended workspace test command from the contributor guide ## Are there any user-facing changes? This adds logical-plan and builder APIs for ASOF joins. SQL syntax, DataFrame APIs, and plan serialization are intentionally left to dependent stack PRs. Floating equality keys remain rejected by the merged physical operator unless the independent follow-up #24375 is also included. As with any new public `LogicalPlan` variant, downstream exhaustive matches must add an arm. The variant is appended so existing variants retain their `PartialOrd` ordering; maintainers should still treat the enum addition as a Rust source-compatibility break. This PR can be reviewed independently now that #23828 has merged. The optimization follow-ups #24799 and #24801 are not required by the core ASOF stack.

## Which issue does this PR close? - Part of #318. - Umbrella PR: #23738. - Builds on #23829 and #23828, both merged. ## Rationale for this change This is the SQL frontend layer of the ASOF JOIN stack. It adds syntax and unparsing on top of the merged physical and logical contracts. The PR now contains only the isolated SQL frontend diff. ## What changes are included in this PR? - Plan `ASOF JOIN ... MATCH_CONDITION (...)` with optional `ON` or `USING` equality keys. - Reject unsupported match shapes and non-equality `ON` predicates. - Unparse ASOF joins while preserving right-side candidate preselection and nested join scope. - Document the supported SQL syntax and semantics, including qualified `USING` keys, left-partitioned broadcast execution, the full-right memory requirement, repeated scans, and the absence of spill/repartitioned ASOF. - Add SQL integration and sqllogictest coverage for all four match directions, coercion, equality-free joins, USING, invalid contracts, EXPLAIN, boundedness, and optimized-plan round trips. - Verify the broadcast topology with a multi-partition left input: left partitioning is preserved while the right input is single-partitioned. ## Are these changes tested? Yes: - `cargo fmt --all` - `./ci/scripts/doc_prettier_check.sh --write --allow-dirty` - `cargo clippy --all-targets --all-features -- -D warnings` - `cargo test -p datafusion --test core_integration asof --all-features` - `cargo test -p datafusion-sqllogictest --test sqllogictests --all-features -- asof_join` - The extended workspace test command from the contributor guide ## Are there any user-facing changes? Users can express Snowflake-style ASOF joins in SQL with `MATCH_CONDITION`, optional equality keys, and `<`, `<=`, `>`, or `>=` match directions. With `USING`, wildcard output exposes one unqualified key while both qualified input keys remain addressable. The user guide also documents the initial broadcast strategy and its memory/no-spill limitations. #23829 and #23828 are merged. This is the next core layer in the ASOF stack and does not depend on the optional floating-point follow-up #24375.

* feat: add ASOF join physical operator (apache#23828) ## Which issue does this PR close? - Part of apache#318. - Umbrella PR: apache#23738. - Follow-up for floating-point equality keys: apache#24375. ## Rationale for this change This is the first layer of the ASOF JOIN stack. It establishes a broadcast-based physical execution contract independently so later floating-point equality, logical-plan, SQL, DataFrame, and serialization changes can be reviewed as smaller follow-up PRs. The initial implementation deliberately favors the simpler broadcast design: the right input must fit in memory and each left partition scans the shared right-side batches. A repartitioned implementation can be evaluated separately without changing the ASOF semantics introduced here. Floating-point equality keys are rejected in this base layer because Arrow's required sort order distinguishes `-0.0` from `+0.0` while join equality does not. apache#24375 adds the required ordering normalization as an independently reviewable layer. ## What changes are included in this PR? - Add `AsOfJoinExec` for left-preserving, Snowflake-style ASOF semantics. - Coalesce and collect the ordered right input once, then share it across all left partitions. - Keep the left input partitioned so each partition can scan independently and preserve the left-side output partitioning. - Preserve merge state across input and output batch boundaries. - Reserve each retained Arrow buffer exactly once, including when right-side batches are zero-copy slices, and expose build, match, and output metrics. - Define output properties and statistics for the broadcast execution model. - Reject floating-point equality keys until apache#24375 supplies a sort/equality contract that handles signed zero correctly. - Add physical operator tests covering match directions, equality groups, batch boundaries, unmatched rows, invalid contracts, shared-buffer memory accounting, multi-partition broadcast execution, and float-key rejection. ## Are these changes tested? Yes: - `cargo fmt --all` - `cargo clippy --all-targets --all-features -- -D warnings` - `cargo test -p datafusion-physical-plan joins::asof_join --all-features` - Extended workspace tests from the contributor guide - FFI integration tests ## Are there any user-facing changes? This adds a new physical operator API. The base operator deliberately rejects floating-point equality keys; apache#24375 adds full Float16, Float32, and Float64 support. SQL and DataFrame APIs are left to later dependent PRs. --------- Co-authored-by: Yongting You <2010youy01@gmail.com> * feat: add ASOF join logical semantics (apache#23829) - Part of apache#318. - Umbrella PR: apache#23738. - Depends on apache#23828 (merged). This is the logical-planning layer of the ASOF JOIN stack. It defines the logical contract and planner behavior separately from the SQL frontend and serialization formats. logical layer. It no longer depends on the optional floating-point follow-up - Add `LogicalPlan::AsOfJoin`, `AsOfJoin`, and `AsOfMatch`. - Validate deterministic expressions, input ownership, supported match operators, equality-key types, and USING constraints. - Add `LogicalPlanBuilder` entry points and schema construction that preserves both qualified `USING` keys while exposing one unqualified wildcard key. - Integrate ASOF joins with tree transforms, display, type coercion, projection pruning, row bounds, and physical planning. - Plan logical ASOF joins to the broadcast-based `AsOfJoinExec` from - Fail closed at proto, SQL unparser, and Substrait boundaries until their owning stack layers add explicit support. - Defer ASOF-specific functional-dependency refinement to apache#24799 and filter pushdown to apache#24801 so each optimization can be reviewed independently. Yes: - `cargo fmt --all` - `cargo clippy --all-targets --all-features -- -D warnings` - `cargo test -p datafusion-expr min_rows_of_joins --all-features` - `cargo test -p datafusion-substrait asof_join_fails_closed_until_substrait_has_an_extension --all-features` - The extended workspace test command from the contributor guide This adds logical-plan and builder APIs for ASOF joins. SQL syntax, DataFrame APIs, and plan serialization are intentionally left to dependent stack PRs. Floating equality keys remain rejected by the merged physical operator unless the independent follow-up apache#24375 is also included. As with any new public `LogicalPlan` variant, downstream exhaustive matches must add an arm. The variant is appended so existing variants retain their `PartialOrd` ordering; maintainers should still treat the enum addition as a Rust source-compatibility break. This PR can be reviewed independently now that apache#23828 has merged. The optimization follow-ups apache#24799 and apache#24801 are not required by the core ASOF stack. * feat: support ASOF JOIN SQL (apache#23830) - Part of apache#318. - Umbrella PR: apache#23738. - Builds on apache#23829 and apache#23828, both merged. This is the SQL frontend layer of the ASOF JOIN stack. It adds syntax and unparsing on top of the merged physical and logical contracts. The PR now contains only the isolated SQL frontend diff. - Plan `ASOF JOIN ... MATCH_CONDITION (...)` with optional `ON` or `USING` equality keys. - Reject unsupported match shapes and non-equality `ON` predicates. - Unparse ASOF joins while preserving right-side candidate preselection and nested join scope. - Document the supported SQL syntax and semantics, including qualified `USING` keys, left-partitioned broadcast execution, the full-right memory requirement, repeated scans, and the absence of spill/repartitioned ASOF. - Add SQL integration and sqllogictest coverage for all four match directions, coercion, equality-free joins, USING, invalid contracts, EXPLAIN, boundedness, and optimized-plan round trips. - Verify the broadcast topology with a multi-partition left input: left partitioning is preserved while the right input is single-partitioned. Yes: - `cargo fmt --all` - `./ci/scripts/doc_prettier_check.sh --write --allow-dirty` - `cargo clippy --all-targets --all-features -- -D warnings` - `cargo test -p datafusion --test core_integration asof --all-features` - `cargo test -p datafusion-sqllogictest --test sqllogictests --all-features -- asof_join` - The extended workspace test command from the contributor guide Users can express Snowflake-style ASOF joins in SQL with `MATCH_CONDITION`, optional equality keys, and `<`, `<=`, `>`, or `>=` match directions. With `USING`, wildcard output exposes one unqualified key while both qualified input keys remain addressable. The user guide also documents the initial broadcast strategy and its memory/no-spill limitations. stack and does not depend on the optional floating-point follow-up apache#24375. --------- Co-authored-by: Xuanwo <github@xuanwo.io> Co-authored-by: Yongting You <2010youy01@gmail.com>

## Which issue does this PR close? - Part of apache#318. - Umbrella PR: apache#23738. - All required ASOF feature layers (apache#23828 through apache#23832) are merged. ## Rationale for this change This is the final core benchmark layer of the ASOF JOIN stack. It uses the standard SQL benchmark framework so the workloads share the existing runner, result format, and Criterion mode. ## What changes are included in this PR? - Add an `asof_join` SQL benchmark suite, runnable with `./bench.sh run asof_join` or `benchmark_runner`. - Add seven workloads covering relative input sizes, available ordering, equality-group cardinality and skew, match direction, and payload width. - Describe each workload and its explicit ASOF match semantics in the query. - Generate pre-sorted Parquet inputs under `DATA_DIR/asof_join` with `./bench.sh data asof_join`. The benchmark load hook only registers them with `WITH ORDER`, keeping data generation and input sorting outside the measured query. - Assert that every workload plans to `AsOfJoinExec`. ## Are these changes tested? Yes: - `cargo fmt --all` - `./ci/scripts/doc_prettier_check.sh --write --allow-dirty` - `cargo clippy --all-targets --all-features -- -D warnings` - `cargo test -p datafusion-benchmarks --lib --bins` - `DATA_DIR=/tmp/asof_join CARGO_COMMAND='cargo run' ./bench.sh data asof_join` - `DATA_DIR=/tmp/asof_join CARGO_COMMAND='cargo run' ./bench.sh run asof_join 7` - `cargo run -p datafusion-benchmarks --bin benchmark_runner -- asof_join --iterations 1 --partitions 4 --path /tmp/asof_join` - `cargo run -p datafusion-benchmarks --bin benchmark_runner -- asof_join --query 7 --partitions 4 --path /tmp/asof_join --criterion` The pre-sorted workload's physical plan contains `AsOfJoinExec` without an input `SortExec`. These smoke runs validate the suite and are not presented as performance claims. ## Are there any user-facing changes? This adds ASOF workloads to the repository benchmark tooling. It does not change query semantics or runtime behavior. This is the final core item tracked by apache#23738. It does not depend on the optional floating-point follow-up apache#24375.

## Which issue does this PR close? - Part of #318. - Umbrella PR: apache#23738. - Follow-up to the merged physical operator in apache#23828. ## Rationale for this change `AsOfJoinExec` already supports an internal output projection, but physical projection pushdown cannot currently populate it. Queries that need only a subset of ASOF output columns therefore materialize unused join columns before the outer projection removes them. ## What changes are included in this PR? - Implement the standard physical projection-embedding hook for `AsOfJoinExec`. - Preserve empty-projection row counts and decline unsafe repeated embedding. - Preserve projected plan properties, including dropping output ordering when its sort expressions are no longer available. - Add focused execution coverage and update the affected ASOF physical-plan expectations. ## Are these changes tested? Yes. The focused ASOF tests, the complete SQL logic test suite, Clippy with all targets and features, and the extended workspace test command from the contributor guide pass. I also compared the merged benchmark Q03 on a dedicated EC2 `c7i.4xlarge` with four partitions. Each revision used its own release build, three warmups, and 40 measured iterations in ABBA order. | Revision | Median | | --- | ---: | | `main` (`133111f43`) | 107.6 ms | | This PR (`72ee47918`) | 106.1 ms | That is a modest 1.3% improvement on Q03. The deterministic benefit is visible in the physical plan: the required output columns are embedded in `AsOfJoinExec`, so unused join columns are not constructed. A residual `ProjectionExec` is retained only when aliases or expressions still require it. ## Are there any user-facing changes? No SQL or logical API changes. Physical plans can avoid materializing ASOF output columns that are not required downstream.

feat: add ASOF join physical operator

e546de5

github-actions Bot added the physical-plan Changes to the physical-plan crate label Jul 23, 2026

Xuanwo mentioned this pull request Jul 23, 2026

feat: add ASOF join logical semantics #23829

Merged

This was referenced Jul 23, 2026

feat: support ASOF JOIN SQL #23830

Merged

feat: add ASOF joins to DataFrame API #23831

Merged

feat: serialize ASOF join plans #23832

Merged

bench: add ASOF join benchmarks #23833

Merged

feat: support ASOF joins #23738

Closed

Xuanwo added 5 commits July 23, 2026 14:30

Merge remote-tracking branch 'origin/main' into xuanwo/asof-physical

68ac8ec

Merge remote-tracking branch 'origin/main' into xuanwo/asof-physical

73fd599

feat: broadcast ASOF right input

43110a3

fix: account shared ASOF build buffers once

339d785

perf: compare ASOF keys without scalar materialization

34552d1

2010YOUY01 reviewed Jul 31, 2026

View reviewed changes

refactor: simplify ASOF physical operator

a127baa

Xuanwo marked this pull request as ready for review August 3, 2026 04:13

Xuanwo requested a review from 2010YOUY01 August 3, 2026 04:14

2010YOUY01 reviewed Aug 10, 2026

View reviewed changes

Update datafusion/physical-plan/src/joins/asof_join.rs

450bffb

Co-authored-by: Yongting You <2010youy01@gmail.com>

github-actions Bot added the auto detected api change Auto detected API change label Aug 11, 2026

refactor: align ASOF physical operator with join conventions

4aa77dc

Xuanwo added 3 commits August 11, 2026 23:26

Merge remote-tracking branch 'origin/main' into xuanwo/asof-physical

0dee525

fix: expose ASOF physical expressions

00ae56b

Merge remote-tracking branch 'origin/main' into xuanwo/asof-physical

70b8145

github-actions Bot removed the auto detected api change Auto detected API change label Aug 11, 2026

Xuanwo requested a review from 2010YOUY01 August 13, 2026 16:21

Xuanwo added 2 commits August 14, 2026 23:53

Merge remote-tracking branch 'origin/main' into xuanwo/asof-physical

f6697b9

fix: reject floating ASOF equality keys

5248402

Xuanwo mentioned this pull request Aug 14, 2026

feat: support floating-point ASOF equality keys #24375

Open

jayzhan211 reviewed Aug 16, 2026

View reviewed changes

Comment thread datafusion/physical-plan/src/joins/asof_join.rs Outdated

refactor: make ASOF join state a stream

5201c34

Xuanwo requested a review from jayzhan211 August 16, 2026 17:09

2010YOUY01 approved these changes Aug 17, 2026

View reviewed changes

jayzhan211 approved these changes Aug 18, 2026

View reviewed changes

fix: report ASOF left input order preservation

5c43ae3

2010YOUY01 added this pull request to the merge queue Aug 20, 2026

Merged via the queue into apache:main with commit 97ba4c9 Aug 20, 2026
40 checks passed

jayzhan211 mentioned this pull request Aug 26, 2026

refactor(asof_join): enhance match value handling and comparator caching #24519

Merged

Xuanwo mentioned this pull request Sep 2, 2026

perf: embed projections in ASOF joins #24873

Merged

osipovartem mentioned this pull request Sep 13, 2026

feat: backport ASOF JOIN to DF55 Embucket/datafusion#86

Merged

nuno-faria added the v56.0.0 label Oct 3, 2026

-    fn maintains_input_order(&self) -> Vec<bool> {
+        // ASOF emits exactly one row for each left row and never reorders the
+        // left input. The right input is scanned independently.
+        vec![true, false]

Conversation

Xuanwo commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

codecov-commenter commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

2010YOUY01 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

2010YOUY01 commented Aug 6, 2026

Uh oh!

Xuanwo commented Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

2010YOUY01 left a comment

Choose a reason for hiding this comment

Need to do before merging

Optional suggestions

Uh oh!

Uh oh!

Uh oh!

2010YOUY01 Aug 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Xuanwo commented Aug 11, 2026

Uh oh!

Xuanwo commented Aug 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jayzhan211 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Xuanwo Aug 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

2010YOUY01 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Xuanwo commented Aug 17, 2026

Uh oh!

Xuanwo commented Jul 23, 2026 •

edited

Loading

codecov-commenter commented Jul 23, 2026 •

edited

Loading

Xuanwo commented Aug 7, 2026 •

edited

Loading

2010YOUY01 Aug 10, 2026 •

edited

Loading

Xuanwo commented Aug 14, 2026 •

edited

Loading

Xuanwo Aug 16, 2026 •

edited

Loading