Skip to content

Support planning logical range repartitioning #22786

Closed

Description

@gene-bordegaray

Will Update With More Depth

Part of #22395.

DataFusion now has a logical Partitioning::Range representation, but LogicalPlan::Repartition with range partitioning is rejected during physical planning.

Add planner support to lower logical range repartitioning to an appropriate physical plan once range repartition execution support is available.

Activity

  1. saadtajwar commented on Jun 23, 2026

    @saadtajwar
    Contributor

    Hey @gene-bordegaray ! If this is still open I'm happy to take it and start digging around!

  2. saadtajwar commented on Jun 23, 2026

    @saadtajwar
    Contributor

    take

  3. saadtajwar commented on Jun 23, 2026

    @saadtajwar
    Contributor

    @gene-bordegaray Hmm, I definitely could be mistaken here but it looks like in the below that logical range repartitioning is now supported during physical planning:

        async fn map_logical_node_to_physical(...) {
    ....
                        LogicalPartitioning::Range(range) => {
                            let sort_exprs = create_physical_sort_exprs(
                                range.ordering(),
                                input_dfschema,
                                execution_props,
                            )?;
                            let ordering = LexOrdering::new(sort_exprs).ok_or_else(|| {
                                internal_datafusion_err!(
                                    "Range repartitioning requires non-empty ordering"
                                )
                            })?;
                            Partitioning::Range(RangePartitioning::try_new(
                                ordering,
                                range.split_points().to_vec(),
                            )?)
                        }
    ...
    }
    

    With the test physical_planner::tests::logical_range_repartition_plans_output_partitioning passing as well! Is the above what this issue was encompassing, or is there something else that I might be missing here? Thanks again for the help! 😁

  4. gene-bordegaray commented on Jun 23, 2026

    @gene-bordegaray
    ContributorAuthor

    yes forgot to close this, but the actual physical implementation for repartitioning for Range is not implemented if you would like to look at that @saadtajwar 👍

  5. gene-bordegaray commented on Jun 23, 2026

    @gene-bordegaray
    ContributorAuthor

    completed in #22777

  6. saadtajwar commented on Jun 23, 2026

    @saadtajwar
    Contributor

    yes forgot to close this, but the actual physical implementation for repartitioning for Range is not implemented if you would like to look at that @saadtajwar 👍

    Sounds good, yeah for sure! I'll make an issue for that if there isn't one already

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions