Skip to content

Conversation

@hubcio
Copy link
Contributor

@hubcio hubcio commented Jan 12, 2026

Centralizing metadata into a single LeftRight store provides lock-free
reads across shards and eliminates coordination complexity inherent in
the distributed slab approach. Shards now read from a shared snapshot
while shard 0 exclusively handles writes, ensuring consistency without
per-read synchronization.

@hubcio hubcio force-pushed the global-metadata-leftright branch 28 times, most recently from bc75595 to f7b7bc6 Compare January 15, 2026 21:01
@hubcio hubcio force-pushed the global-metadata-leftright branch 2 times, most recently from 61eae14 to c8b6bdc Compare January 20, 2026 12:59
@hubcio hubcio force-pushed the global-metadata-leftright branch 7 times, most recently from 5d443ab to 12bf4f9 Compare January 21, 2026 11:41
@hubcio hubcio changed the title refactor(server): WIP, don't merge: replace slab ECS with LeftRight-based global metadata refactor(server): replace slab ECS with LeftRight-based global metadata Jan 21, 2026
@hubcio hubcio force-pushed the global-metadata-leftright branch 4 times, most recently from ba0e23c to d971aad Compare January 21, 2026 15:05
@hubcio hubcio force-pushed the global-metadata-leftright branch 2 times, most recently from 8a1a24c to 2d20028 Compare January 23, 2026 10:14
@numinnex
Copy link
Contributor

Overall, this looks good. If you don't have anything else to add there (tests or anything), I think we could merge this.

@hubcio
Copy link
Contributor Author

hubcio commented Jan 23, 2026

@krishvishal @spetz @mmodzelewski mind taking a look?

Centralizing metadata into a single LeftRight store provides lock-free
reads across shards and eliminates coordination complexity inherent in
the distributed slab approach. Shards now read from a shared snapshot
while shard 0 exclusively handles writes, ensuring consistency without
per-read synchronization.
@hubcio hubcio force-pushed the global-metadata-leftright branch from 2d20028 to 0e81d8f Compare January 23, 2026 12:13
Copy link
Contributor

@krishvishal krishvishal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the add_stream we are currently using 0 as the sentinel value but in add_topic we are using usize:MAX as the sentinel value. This can be inconsistent.

pub fn add_stream(&mut self, meta: StreamMeta) -> StreamId {
let assigned_id = Arc::new(AtomicUsize::new(0));

@hubcio hubcio force-pushed the global-metadata-leftright branch from 0e81d8f to 6588e1a Compare January 23, 2026 16:39
@hubcio
Copy link
Contributor Author

hubcio commented Jan 23, 2026

In the add_stream we are currently using 0 as the sentinel value but in add_topic we are using usize:MAX as the sentinel value. This can be inconsistent.

pub fn add_stream(&mut self, meta: StreamMeta) -> StreamId {
let assigned_id = Arc::new(AtomicUsize::new(0));

good catch, fixed

@hubcio hubcio merged commit de6995d into master Jan 26, 2026
55 checks passed
@hubcio hubcio deleted the global-metadata-leftright branch January 26, 2026 12:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants