Repository navigation
Add example implementing filter pushdown #21145
Description
Is your feature request related to a problem or challenge?
@pgwhalen says on https://github.com/apache/datafusion-site/pull/161/changes#r2972139725
As someone who struggled with implementing filter pushdown in a custom table provider before, it would be helpful if there was a section with examples on actually implementing it, just to specifying that the table provider supports it. Going from filters: &[Expr] to something more useful was pretty intimidating for me as a newbie. I ended up learning about LiteralGuarantee::analyze from the pruning_predicate.rs example and that was the big unlock, but as someone not familiar with database internals terminology, it was nonobvious initially.
Describe the solution you'd like
It would be cool to add an example that shows how to use PruningPredicate and LiteralGuarantee to decide among files based on predicates, for example
Describe alternatives you've considered
No response
Additional context
No response
Activity
alamb commentedon Mar 25, 2026
Somewhere in https://github.com/apache/datafusion/tree/main/datafusion-examples/examples/custom_data_source I think would be good
alamb commentedon Mar 25, 2026
Note the example on apache/datafusion-site#161 shows the setup needed, but the idea of this example would be to show how to use pruning prediate / gaurantees to actually implement the filtering
pgwhalen commentedon Mar 26, 2026
I can take a crack at this one over the weekend if that's okay!
alamb commentedon Mar 26, 2026
That would be amazing
I think ensuring you review the existing examples would be good -- and if you can work it in to one of those great and if not a new example would be great
pgwhalen commentedon Mar 30, 2026
take