catalog: order builtin indexes immediately after their ON relations#34856
Draft
teskje wants to merge 1 commit intoMaterializeInc:mainfrom
Draft
catalog: order builtin indexes immediately after their ON relations#34856teskje wants to merge 1 commit intoMaterializeInc:mainfrom
teskje wants to merge 1 commit intoMaterializeInc:mainfrom
Conversation
0c0fd13 to
a39008c
Compare
This commit changes the order of builtin objects in `BUILTIN_STATIC` to move all indexes as far up as possible, i.e. immediately below the objects they index. The `BUILTIN_STATIC` order determines in which order objects are created during bootstrapping, so moving indexes up ensures that they can be used by all other objects depending on the indexed relations. Previously, builtin objects would commonly read data from persist again and duplicate view fragments, instead of using readily available indexes.
a39008c to
9ea31d8
Compare
Contributor
Author
|
It actually doesn't work 🙈 From Slack:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR changes the order of builtin objects in
BUILTIN_STATICto move all indexes as far up as possible, i.e. immediately below the objects they index. TheBUILTIN_STATICorder determines in which order objects are created during bootstrapping, so moving indexes up ensures that they can be used by all other objects depending on the indexed relations. Previously, builtin objects would commonly read data from persist again and duplicate view fragments, instead of using readily available indexes.On main, this change reduces the number of dataflow operators in
mz_catalog_serverfrom 11863 to 9948 and the number of arrangements from 842 to 763.Motivation
mz_catalog_serverhas a bunch of dataflows that don't make use of available indexes. Slack thread.Tips for reviewer
Checklist
$T ⇔ Proto$Tmapping (possibly in a backwards-incompatible way), then it is tagged with aT-protolabel.