-
Notifications
You must be signed in to change notification settings - Fork 258
refactor(server): replace slab ECS with LeftRight-based global metadata #2555
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
bc75595 to
f7b7bc6
Compare
61eae14 to
c8b6bdc
Compare
5d443ab to
12bf4f9
Compare
ba0e23c to
d971aad
Compare
8a1a24c to
2d20028
Compare
|
Overall, this looks good. If you don't have anything else to add there (tests or anything), I think we could merge this. |
|
@krishvishal @spetz @mmodzelewski mind taking a look? |
Centralizing metadata into a single LeftRight store provides lock-free reads across shards and eliminates coordination complexity inherent in the distributed slab approach. Shards now read from a shared snapshot while shard 0 exclusively handles writes, ensuring consistency without per-read synchronization.
2d20028 to
0e81d8f
Compare
krishvishal
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In the add_stream we are currently using 0 as the sentinel value but in add_topic we are using usize:MAX as the sentinel value. This can be inconsistent.
iggy/core/server/src/metadata/writer.rs
Lines 68 to 69 in 0e81d8f
| pub fn add_stream(&mut self, meta: StreamMeta) -> StreamId { | |
| let assigned_id = Arc::new(AtomicUsize::new(0)); |
0e81d8f to
6588e1a
Compare
good catch, fixed |
Centralizing metadata into a single LeftRight store provides lock-free
reads across shards and eliminates coordination complexity inherent in
the distributed slab approach. Shards now read from a shared snapshot
while shard 0 exclusively handles writes, ensuring consistency without
per-read synchronization.