Repository navigation
minor: Refactor expression microbenchmarks to remove duplicate code - #2956
Conversation
|
@coderfender could you review? |
| withSQLConf(CometConf.COMET_ENABLED.key -> "true") { | ||
| spark.sql(query).noop() | ||
| } | ||
| } |
There was a problem hiding this comment.
Minor : Would we be needing exec and scan exec for all types of benchmarks @andygrove ?
There was a problem hiding this comment.
Yes, I only updated expression benchmarks in this PR but will follow up for the other ones (shuffle, exec, aggregate) in separate PRs in the future but those ones are a little more involved. My short term goal is to add benchmarks for all expressions, so wanted to refactor the expression benchmarks before doing that.
|
Might want to fix unused imports to pass CI |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2956 +/- ##
============================================
+ Coverage 56.12% 59.60% +3.47%
- Complexity 976 1381 +405
============================================
Files 119 167 +48
Lines 11743 15493 +3750
Branches 2251 2568 +317
============================================
+ Hits 6591 9234 +2643
- Misses 4012 4959 +947
- Partials 1140 1300 +160 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
comphead
left a comment
There was a problem hiding this comment.
Thanks @andygrove nice clean up
Which issue does this PR close?
Part of #2955
Rationale for this change
Each expression microbenchmark repeats the following code, but with differences in naming:
This PR moves this logic into a base class and re-uses it across all existing expression microbenchmarks.
What changes are included in this PR?
How are these changes tested?