Skip to content

FilterPushdown will generate duplicate exprs #3914

Closed

Description

@waynexia

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
The plan used to (before #3841) be like

https://github.com/apache/arrow-datafusion/blob/a9ee67471d1ab0ad11f12d024315b3a824d6101f/benchmarks/expected-plans/q21.txt#L15

Which contains some duplicate exprs, see the file diff for highlights.

And in #3841 @Dandandan run some optimize rules one more time and got the duplicate exprs optimized out. But in #3861 the duplicate exprs will cause another problem when combine with CommonSubexpressionEliminate rule.

Describe the solution you'd like

Besides let other rules to do further optimization (this is useful to make more optimizations), I think it's also important to generate a "good enough" plan from a rule itself (i.e., remove duplications like this in FilterPushdown). So we can cut the dependent graph among individual plans (which is hard to maintain...).

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Activity

  1. alamb commented on Oct 2, 2023

    @alamb
    Contributor

    I wonder if this is still an issue

  2. alamb commented on Feb 12, 2026

    @alamb
    Contributor

    The current plan has no duplicate filters

    19)----------------------Filter: orders.o_orderstatus = Utf8View("F")
    20)------------------------TableScan: orders projection=[o_orderkey, o_orderstatus], partial_filters=[orders.o_orderstatus = Utf8View("F")]

    So I am going to claim this one is fixed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions