Skip to content

Parquet pruneing incorrectly handles IS NOT NULL predicate #10049

Closed

Description

@Ted-Jiang
          > Thanks @Ted-Jiang -- the support for `IS NOT NULL` is also quite neat.

@alamb sorry there is a bug here i will fix this it should

/// If set `with_not` to true: which means is not null
/// datafusion use false flag to prune unit (row group, page ..)
/// Given an expression reference to `expr`, if `expr` is a column expression,
/// returns a pruning expression in terms of IsNotNull that will evaluate to true
/// if the column may contain any nonnull values, and false if definitely does not contain
/// nonnull values which any null values.

Originally posted by @Ted-Jiang in #9989 (comment)

Activity

  1. Ted-Jiang commented on Apr 12, 2024

    @Ted-Jiang
    MemberAuthor

    I will give a fix soon

  2. self-assigned this
    on Apr 12, 2024
  3. added
    bugSomething isn't working
    regressionSomething that used to work no longer does
    on Apr 12, 2024
  4. changed the title [-]Fix parquet prune handle IS NOT NULL[/-] [+]Parquet pruneing incorrectly handles `IS NOT NULL` predicate[/+] on Apr 12, 2024
  5. alamb commented on Apr 12, 2024

    @alamb
    Contributor

    Note this bug was not in any release of DataFusion yet

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingregressionSomething that used to work no longer does

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions