feat: Add WebSocket and real-time collaboration protocol specifications#376
feat: Add WebSocket and real-time collaboration protocol specifications#376
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
- Created websocket.zod.ts with comprehensive WebSocket event protocol - Created collaboration.zod.ts with OT, CRDT, cursor sharing, and awareness - Added comprehensive test coverage for both protocols - Updated exports in index files - All tests passing and build successful Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
|
This PR is very large. Consider breaking it into smaller PRs for easier review. |
There was a problem hiding this comment.
Pull request overview
Adds protocol specifications for WebSocket-based realtime messaging and collaboration, including generated JSON Schema and documentation pages.
Changes:
- Introduces
packages/spec/src/api/websocket.zod.tswith subscription patterns, filters, presence/cursor/edit message envelopes, and connection config. - Introduces
packages/spec/src/system/collaboration.zod.tswith OT/CRDT/cursor/awareness/session schemas. - Adds Vitest coverage, docs reference pages, and generated JSON-schema artifacts; updates barrel exports.
Reviewed changes
Copilot reviewed 67 out of 67 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/spec/src/system/index.ts | Re-exports the new collaboration protocol from the system package entrypoint. |
| packages/spec/src/system/collaboration.zod.ts | Adds OT/CRDT/cursor/awareness/session Zod schemas for collaboration. |
| packages/spec/src/system/collaboration.test.ts | Adds Vitest validation tests for collaboration schemas. |
| packages/spec/src/api/websocket.zod.ts | Adds WebSocket message, subscription/filter, presence/cursor/edit, and config schemas. |
| packages/spec/src/api/websocket.test.ts | Adds Vitest validation tests for WebSocket schemas. |
| packages/spec/src/api/index.ts | Re-exports the new websocket protocol from the API package entrypoint. |
| packages/spec/json-schema/system/VectorClock.json | Generated JSON schema for VectorClock. |
| packages/spec/json-schema/system/UserActivityStatus.json | Generated JSON schema for UserActivityStatus. |
| packages/spec/json-schema/system/TextCRDTState.json | Generated JSON schema for TextCRDTState. |
| packages/spec/json-schema/system/TextCRDTOperation.json | Generated JSON schema for TextCRDTOperation. |
| packages/spec/json-schema/system/PNCounter.json | Generated JSON schema for PNCounter. |
| packages/spec/json-schema/system/OTTransformResult.json | Generated JSON schema for OTTransformResult. |
| packages/spec/json-schema/system/OTOperationType.json | Generated JSON schema for OTOperationType. |
| packages/spec/json-schema/system/OTOperation.json | Generated JSON schema for OTOperation. |
| packages/spec/json-schema/system/OTComponent.json | Generated JSON schema for OTComponent. |
| packages/spec/json-schema/system/ORSetElement.json | Generated JSON schema for ORSetElement. |
| packages/spec/json-schema/system/ORSet.json | Generated JSON schema for ORSet. |
| packages/spec/json-schema/system/LWWRegister.json | Generated JSON schema for LWWRegister. |
| packages/spec/json-schema/system/GCounter.json | Generated JSON schema for GCounter. |
| packages/spec/json-schema/system/CursorUpdate.json | Generated JSON schema for CursorUpdate. |
| packages/spec/json-schema/system/CursorStyle.json | Generated JSON schema for CursorStyle. |
| packages/spec/json-schema/system/CursorSelection.json | Generated JSON schema for CursorSelection. |
| packages/spec/json-schema/system/CursorColorPreset.json | Generated JSON schema for CursorColorPreset. |
| packages/spec/json-schema/system/CounterOperation.json | Generated JSON schema for CounterOperation. |
| packages/spec/json-schema/system/CollaborativeCursor.json | Generated JSON schema for CollaborativeCursor. |
| packages/spec/json-schema/system/CollaborationSessionConfig.json | Generated JSON schema for CollaborationSessionConfig. |
| packages/spec/json-schema/system/CollaborationSession.json | Generated JSON schema for CollaborationSession. |
| packages/spec/json-schema/system/CollaborationMode.json | Generated JSON schema for CollaborationMode. |
| packages/spec/json-schema/system/CRDTType.json | Generated JSON schema for CRDTType. |
| packages/spec/json-schema/system/CRDTState.json | Generated JSON schema for CRDTState. |
| packages/spec/json-schema/system/CRDTMergeResult.json | Generated JSON schema for CRDTMergeResult. |
| packages/spec/json-schema/system/AwarenessUserState.json | Generated JSON schema for AwarenessUserState. |
| packages/spec/json-schema/system/AwarenessUpdate.json | Generated JSON schema for AwarenessUpdate. |
| packages/spec/json-schema/system/AwarenessSession.json | Generated JSON schema for AwarenessSession. |
| packages/spec/json-schema/system/AwarenessEvent.json | Generated JSON schema for AwarenessEvent. |
| packages/spec/json-schema/api/WebSocketPresenceStatus.json | Generated JSON schema for WebSocketPresenceStatus. |
| packages/spec/json-schema/api/WebSocketMessageType.json | Generated JSON schema for WebSocketMessageType. |
| packages/spec/json-schema/api/WebSocketMessage.json | Generated JSON schema for WebSocketMessage union. |
| packages/spec/json-schema/api/WebSocketConfig.json | Generated JSON schema for WebSocketConfig. |
| packages/spec/json-schema/api/UnsubscribeRequest.json | Generated JSON schema for UnsubscribeRequest. |
| packages/spec/json-schema/api/UnsubscribeMessage.json | Generated JSON schema for UnsubscribeMessage. |
| packages/spec/json-schema/api/SubscribeMessage.json | Generated JSON schema for SubscribeMessage. |
| packages/spec/json-schema/api/PresenceUpdate.json | Generated JSON schema for PresenceUpdate. |
| packages/spec/json-schema/api/PresenceState.json | Generated JSON schema for PresenceState. |
| packages/spec/json-schema/api/PresenceMessage.json | Generated JSON schema for PresenceMessage. |
| packages/spec/json-schema/api/PongMessage.json | Generated JSON schema for PongMessage. |
| packages/spec/json-schema/api/PingMessage.json | Generated JSON schema for PingMessage. |
| packages/spec/json-schema/api/FilterOperator.json | Generated JSON schema for FilterOperator. |
| packages/spec/json-schema/api/EventSubscription.json | Generated JSON schema for EventSubscription. |
| packages/spec/json-schema/api/EventPattern.json | Generated JSON schema for EventPattern. |
| packages/spec/json-schema/api/EventMessage.json | Generated JSON schema for EventMessage. |
| packages/spec/json-schema/api/EventFilterCondition.json | Generated JSON schema for EventFilterCondition. |
| packages/spec/json-schema/api/EventFilter.json | Generated JSON schema for EventFilter. |
| packages/spec/json-schema/api/ErrorMessage.json | Generated JSON schema for ErrorMessage. |
| packages/spec/json-schema/api/EditOperationType.json | Generated JSON schema for EditOperationType. |
| packages/spec/json-schema/api/EditOperation.json | Generated JSON schema for EditOperation. |
| packages/spec/json-schema/api/EditMessage.json | Generated JSON schema for EditMessage. |
| packages/spec/json-schema/api/DocumentState.json | Generated JSON schema for DocumentState. |
| packages/spec/json-schema/api/CursorPosition.json | Generated JSON schema for CursorPosition. |
| packages/spec/json-schema/api/CursorMessage.json | Generated JSON schema for CursorMessage. |
| packages/spec/json-schema/api/AckMessage.json | Generated JSON schema for AckMessage. |
| content/docs/references/system/meta.json | Adds the collaboration docs page to system references navigation. |
| content/docs/references/system/index.mdx | Adds a navigation card for the collaboration protocol docs. |
| content/docs/references/system/collaboration.mdx | Adds a reference page for collaboration schemas. |
| content/docs/references/api/websocket.mdx | Adds a reference page for websocket schemas. |
| content/docs/references/api/meta.json | Adds the websocket docs page to API references navigation. |
| content/docs/references/api/index.mdx | Adds a navigation card for the websocket protocol docs. |
| export const EventFilterCondition = z.object({ | ||
| field: z.string().describe('Field path to filter on (supports dot notation, e.g., "user.email")'), | ||
| operator: FilterOperator.describe('Comparison operator'), | ||
| value: z.any().optional().describe('Value to compare against (not needed for "exists" operator)'), | ||
| }); |
There was a problem hiding this comment.
EventFilterCondition allows value to be omitted for operators like eq, gt, in, etc. This contradicts the field description and will accept structurally invalid filters. Consider modeling this as a discriminated union on operator (e.g., require value for all operators except exists, and constrain in/nin to arrays, regex to string/RegExp-like inputs).
| export const EventSubscriptionSchema = z.object({ | ||
| subscriptionId: z.string().uuid().describe('Unique subscription identifier'), | ||
| events: z.array(EventPatternSchema).describe('Event patterns to subscribe to (supports wildcards, e.g., "record.*", "user.created")'), | ||
| objects: z.array(z.string()).optional().describe('Object names to filter events by (e.g., ["account", "contact"])'), | ||
| filters: EventFilterSchema.optional().describe('Advanced filter conditions for event payloads'), | ||
| channels: z.array(z.string()).optional().describe('Channel names for scoped subscriptions'), | ||
| }); |
There was a problem hiding this comment.
events in EventSubscriptionSchema is not constrained to be non-empty, so { events: [] } will currently validate. If at least one event pattern is required for a meaningful subscription, add a .min(1) on the array (and align tests accordingly).
| export const EditOperationType = z.enum([ | ||
| 'insert', // Insert text at position | ||
| 'delete', // Delete text from range | ||
| 'replace', // Replace text in range | ||
| ]); | ||
|
|
||
| export type EditOperationType = z.infer<typeof EditOperationType>; | ||
|
|
||
| /** | ||
| * Edit Operation Schema | ||
| * Represents a single edit operation on a document | ||
| * Supports Operational Transformation (OT) for conflict resolution | ||
| */ | ||
| export const EditOperationSchema = z.object({ | ||
| operationId: z.string().uuid().describe('Unique operation identifier'), | ||
| documentId: z.string().describe('Document identifier'), | ||
| userId: z.string().describe('User who performed the edit'), | ||
| sessionId: z.string().uuid().describe('Session identifier'), | ||
| type: EditOperationType.describe('Type of edit operation'), | ||
| position: z.object({ | ||
| line: z.number().int().nonnegative().describe('Line number (0-indexed)'), | ||
| column: z.number().int().nonnegative().describe('Column number (0-indexed)'), | ||
| }).describe('Starting position of the operation'), | ||
| endPosition: z.object({ | ||
| line: z.number().int().nonnegative(), | ||
| column: z.number().int().nonnegative(), | ||
| }).optional().describe('Ending position (for delete/replace operations)'), | ||
| content: z.string().optional().describe('Content to insert/replace'), | ||
| version: z.number().int().nonnegative().describe('Document version before this operation'), | ||
| timestamp: z.string().datetime().describe('ISO 8601 datetime when operation was created'), | ||
| baseOperationId: z.string().uuid().optional().describe('Previous operation ID this builds upon (for OT)'), | ||
| }); |
There was a problem hiding this comment.
EditOperationSchema doesn’t enforce the shape implied by type: insert can omit content, and delete/replace can omit endPosition (and replace can omit content). Consider switching this to a discriminated union on type with per-operation required/forbidden fields to prevent invalid edit operations from validating.
| export const TextCRDTOperationSchema = z.object({ | ||
| operationId: z.string().uuid().describe('Unique operation identifier'), | ||
| replicaId: z.string().describe('Replica identifier'), | ||
| position: z.number().int().nonnegative().describe('Position in document'), | ||
| insert: z.string().optional().describe('Text to insert'), | ||
| delete: z.number().int().positive().optional().describe('Number of characters to delete'), | ||
| timestamp: z.string().datetime().describe('ISO 8601 datetime of operation'), | ||
| lamportTimestamp: z.number().int().nonnegative().describe('Lamport timestamp for ordering'), | ||
| }); |
There was a problem hiding this comment.
TextCRDTOperationSchema allows both insert and delete to be omitted (and also allows both to be present), which means no-op or ambiguous operations validate. Consider enforcing exactly one of insert or delete (e.g., discriminated union or a .refine XOR constraint).
| export const CRDTType = z.enum([ | ||
| 'lww-register', // Last-Write-Wins Register | ||
| 'g-counter', // Grow-only Counter | ||
| 'pn-counter', // Positive-Negative Counter | ||
| 'g-set', // Grow-only Set | ||
| 'or-set', // Observed-Remove Set | ||
| 'lww-map', // Last-Write-Wins Map | ||
| 'text', // CRDT-based Text (e.g., Yjs, Automerge) | ||
| 'tree', // CRDT-based Tree structure | ||
| 'json', // CRDT-based JSON (e.g., Automerge) | ||
| ]); | ||
|
|
There was a problem hiding this comment.
CRDTType lists values (g-set, lww-map, tree, json) that have no corresponding state schemas and are not included in CRDTStateSchema. This makes the protocol surface area ambiguous. Either add schemas for these CRDT variants (and include them in CRDTStateSchema) or remove them from CRDTType until supported.
| ## TypeScript Usage | ||
|
|
||
| ```typescript | ||
| import { AwarenessEventSchema, AwarenessSessionSchema, AwarenessUpdateSchema, AwarenessUserStateSchema, CRDTMergeResultSchema, CRDTStateSchema, CRDTTypeSchema, CollaborationModeSchema, CollaborationSessionSchema, CollaborationSessionConfigSchema, CollaborativeCursorSchema, CounterOperationSchema, CursorColorPresetSchema, CursorSelectionSchema, CursorStyleSchema, CursorUpdateSchema, GCounterSchema, LWWRegisterSchema, ORSetSchema, ORSetElementSchema, OTComponentSchema, OTOperationSchema, OTOperationTypeSchema, OTTransformResultSchema, PNCounterSchema, TextCRDTOperationSchema, TextCRDTStateSchema, UserActivityStatusSchema, VectorClockSchema } from '@objectstack/spec/system'; |
There was a problem hiding this comment.
The TypeScript import example references schema exports like CRDTTypeSchema, CursorColorPresetSchema, and OTOperationTypeSchema, but in packages/spec/src/system/collaboration.zod.ts the corresponding exports are named CRDTType, CursorColorPreset, and OTOperationType (no Schema suffix). Update the example to match the actual exported names to avoid misleading consumers.
| import { AwarenessEventSchema, AwarenessSessionSchema, AwarenessUpdateSchema, AwarenessUserStateSchema, CRDTMergeResultSchema, CRDTStateSchema, CRDTTypeSchema, CollaborationModeSchema, CollaborationSessionSchema, CollaborationSessionConfigSchema, CollaborativeCursorSchema, CounterOperationSchema, CursorColorPresetSchema, CursorSelectionSchema, CursorStyleSchema, CursorUpdateSchema, GCounterSchema, LWWRegisterSchema, ORSetSchema, ORSetElementSchema, OTComponentSchema, OTOperationSchema, OTOperationTypeSchema, OTTransformResultSchema, PNCounterSchema, TextCRDTOperationSchema, TextCRDTStateSchema, UserActivityStatusSchema, VectorClockSchema } from '@objectstack/spec/system'; | |
| import { AwarenessEventSchema, AwarenessSessionSchema, AwarenessUpdateSchema, AwarenessUserStateSchema, CRDTMergeResultSchema, CRDTStateSchema, CRDTType, CollaborationModeSchema, CollaborationSessionSchema, CollaborationSessionConfigSchema, CollaborativeCursorSchema, CounterOperationSchema, CursorColorPreset, CursorSelectionSchema, CursorStyleSchema, CursorUpdateSchema, GCounterSchema, LWWRegisterSchema, ORSetSchema, ORSetElementSchema, OTComponentSchema, OTOperationSchema, OTOperationType, OTTransformResultSchema, PNCounterSchema, TextCRDTOperationSchema, TextCRDTStateSchema, UserActivityStatusSchema, VectorClockSchema } from '@objectstack/spec/system'; |
| ## TypeScript Usage | ||
|
|
||
| ```typescript | ||
| import { AckMessageSchema, CursorMessageSchema, CursorPositionSchema, DocumentStateSchema, EditMessageSchema, EditOperationSchema, EditOperationTypeSchema, ErrorMessageSchema, EventFilterSchema, EventFilterConditionSchema, EventMessageSchema, EventPatternSchema, EventSubscriptionSchema, FilterOperatorSchema, PingMessageSchema, PongMessageSchema, PresenceMessageSchema, PresenceStateSchema, PresenceUpdateSchema, SubscribeMessageSchema, UnsubscribeMessageSchema, UnsubscribeRequestSchema, WebSocketConfigSchema, WebSocketMessageSchema, WebSocketMessageTypeSchema, WebSocketPresenceStatusSchema } from '@objectstack/spec/api'; |
There was a problem hiding this comment.
The TypeScript import example references schema exports like FilterOperatorSchema, EditOperationTypeSchema, WebSocketMessageTypeSchema, etc., but packages/spec/src/api/websocket.zod.ts exports FilterOperator, EditOperationType, WebSocketMessageType, etc. (no Schema suffix). Update the example to match actual exports to prevent copy/paste errors.
| import { AckMessageSchema, CursorMessageSchema, CursorPositionSchema, DocumentStateSchema, EditMessageSchema, EditOperationSchema, EditOperationTypeSchema, ErrorMessageSchema, EventFilterSchema, EventFilterConditionSchema, EventMessageSchema, EventPatternSchema, EventSubscriptionSchema, FilterOperatorSchema, PingMessageSchema, PongMessageSchema, PresenceMessageSchema, PresenceStateSchema, PresenceUpdateSchema, SubscribeMessageSchema, UnsubscribeMessageSchema, UnsubscribeRequestSchema, WebSocketConfigSchema, WebSocketMessageSchema, WebSocketMessageTypeSchema, WebSocketPresenceStatusSchema } from '@objectstack/spec/api'; | |
| import { AckMessageSchema, CursorMessageSchema, CursorPositionSchema, DocumentStateSchema, EditMessageSchema, EditOperationSchema, EditOperationType, ErrorMessageSchema, EventFilterSchema, EventFilterConditionSchema, EventMessageSchema, EventPatternSchema, EventSubscriptionSchema, FilterOperator, PingMessageSchema, PongMessageSchema, PresenceMessageSchema, PresenceStateSchema, PresenceUpdateSchema, SubscribeMessageSchema, UnsubscribeMessageSchema, UnsubscribeRequestSchema, WebSocketConfigSchema, WebSocketMessageSchema, WebSocketMessageType, WebSocketPresenceStatus } from '@objectstack/spec/api'; |
Implements comprehensive Zod schemas for WebSocket-based real-time communication and collaborative editing, addressing requirements for event subscriptions, filtering, presence tracking, and conflict-free collaboration.
WebSocket Event Protocol (
api/websocket.zod.ts)Event Subscriptions: Wildcard pattern matching (
record.*,*.created) viaEventPatternSchema, distinct from dot-notationEventNameSchemafor actual events.Advanced Filtering: Recursive
EventFilterSchemawith explicitZodTypeannotation to resolve circular reference. Supports 13 operators (eq, ne, gt, gte, lt, lte, in, nin, contains, startsWith, endsWith, exists, regex) with AND/OR/NOT combinators.Presence & Collaboration: Enhanced presence tracking (4 states: online/away/busy/offline) with device detection. Cursor positions with line/column tracking. Edit operations (insert/delete/replace) with version tracking for OT.
Protocol Messages: 10 discriminated union message types with
WebSocketMessageSchemafor type-safe parsing.Real-Time Collaboration Protocol (
system/collaboration.zod.ts)Operational Transformation:
OTComponentSchemadiscriminated union (insert/delete/retain) with attribute support for rich text formatting.CRDT Types:
Cursor Sharing:
CollaborativeCursorSchemawith selection ranges (anchor/focus), visual styling (10 color presets + custom), and typing indicators.Awareness State:
AwarenessUserStateSchematracks user activity (active/idle/viewing/disconnected), current document/view, and session lifecycle events.Collaboration Sessions: Supports 4 modes (ot/crdt/lock/hybrid) with configurable conflict resolution, snapshot intervals, and idle timeouts.
Technical Notes
PresenceStatus→WebSocketPresenceStatusto avoid export collision withrealtime.zod.tsz.infer<>type derivationOriginal prompt
💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.