Migrate Join logic away from traits#668
Merged
frankmcsherry merged 2 commits intoTimelyDataflow:masterfrom Feb 20, 2026
Merged
Conversation
frankmcsherry
commented
Feb 20, 2026
| /// The underlying `Stream<G, BatchWrapper<T::Batch>>` is a much more efficient way to access the data, | ||
| /// and this method should only be used when the data need to be transformed or exchanged, rather than | ||
| /// supplied as arguments to an operator using the same key-value structure. | ||
| pub fn as_vecs(&self) -> VecCollection<G, (Tr::KeyOwn, Tr::ValOwn), Tr::Diff> |
Member
Author
There was a problem hiding this comment.
I forgot to mention I added this. We should use it all over the place instead of as_collection with logic that happens to do the same thing.
antiguru
approved these changes
Feb 20, 2026
Member
antiguru
left a comment
antiguru
left a comment
There was a problem hiding this comment.
Looks good, delta mdbook failures.
frankmcsherry
force-pushed
the
de_vec_join
branch
2 times, most recently
from
a10bedc to
d8dfab3
Compare
frankmcsherry
force-pushed
the
de_vec_join
branch
from
d8dfab3 to
fee2f9f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The logic for
joinhas been provided by traits likeJoinandJoinCore, whose implementations involveVecCollectionmore than we might like. Implementations forVecCollectionhave become inherent methods, leaving the only other implementorArrangedwho gets some inherent methods and some clean-up.There are some very clear breaking changes here, and probably a few more to implement.
If you previously used an arrangement to
joinwith a collection, you'll need to usejoin_coreinstead. Many places usedsemijoinon arrangements and it's just wrong (well, probably correctly used, but only a semijoin because the data happen to have primary keys). These calls also need light changes. Generally, usingjoin_core, which currently still producesVecCollectionoutput, but I'd like to make it easier to avoid that without having to drop all the way tojoin_traces.We should probably delete/deprecate
semijoinandantijoin. They are the right "pattern", but they don't actually guarantee that they do what their name says (e.g. they do not ensure distinctness of keys in their collections). They are also just sugar for 1-2 lines each, with a bunch of boilerplate supporting them.