-
Notifications
You must be signed in to change notification settings - Fork 268
chore: Add checks to microbenchmarks for plan running natively in Comet #3045
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
| } | ||
|
|
||
| /** Runs function `f` with Comet on and off. */ | ||
| final def runWithComet(name: String, cardinality: Long)(f: => Unit): Unit = { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This method is no longer used
| * | ||
| * Based on CometTestBase.findFirstNonCometOperator. | ||
| */ | ||
| protected def findFirstNonCometOperator(plan: SparkPlan): Option[SparkPlan] = { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should we keep one version for this methods to keep in sync the test behavior?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I moved this to a new CometPlanChecker trait to avoid duplication.
We may want to rename this trait in the future if we move more code there.
comphead
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm thanks @andygrove
mbutrovich
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the quick fix, @andygrove!
parthchandra
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3045 +/- ##
============================================
+ Coverage 56.12% 59.55% +3.43%
- Complexity 976 1379 +403
============================================
Files 119 167 +48
Lines 11743 15496 +3753
Branches 2251 2569 +318
============================================
+ Hits 6591 9229 +2638
- Misses 4012 4970 +958
- Partials 1140 1297 +157 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Which issue does this PR close?
Closes #3019
Rationale for this change
If benchmark queries are not running fully natively then it could be due to a bug or incorrect config and we want to know this.
Example:
What changes are included in this PR?
Show a warning if a query does not run fully natively when running
Comet (Scan + Exec).How are these changes tested?