Merge pull request #343 from ably/objects-operation-application

[PUB-1825, PUB-1826] Add spec for applying incoming OBJECT messages

Author: VeskeR

src/objects-features.md

@@ -82,7 +82,8 @@ the latest state.
       that the client SDK must not create a new `LiveMap` instance with
       id `root`; it must only clear the internal data of the existing
       `LiveMap` with id `root`
-    - `(RTO4b3)` The `SyncObjectsPool` must be cleared
+    - `(RTO4b3)` The `SyncObjectsPool` list must be cleared
+    - `(RTO4b5)` The `BufferedObjectOperations` list must be cleared
     - `(RTO4b4)` Perform the actions for objects sync completion as
       described in [RTO5c](#RTO5c)
 - `(RTO5)` The realtime system reserves the right to initiate an objects
@@ -100,7 +101,8 @@ the latest state.
     - `(RTO5a2)` If a new sequence id is sent from Ably, the client
       library must treat it as the start of a new objects sync sequence,
       and any previous in-flight sync must be discarded:
-      - `(RTO5a2a)` The current `SyncObjectsPool` list must be cleared
+      - `(RTO5a2a)` The `SyncObjectsPool` list must be cleared
+      - `(RTO5a2b)` The `BufferedObjectOperations` list must be cleared
     - `(RTO5a3)` If the sequence id matches the previously received
       sequence id, the client library should continue the sync process
     - `(RTO5a4)` The objects sync sequence for that sequence identifier
@@ -111,7 +113,7 @@ the latest state.
       contained within the `ProtocolMessage`
   - `(RTO5b)` During the sync sequence, the
     [`ObjectMessage.object`](../features#TR4r) values from incoming
-    `OBJECT_SYNC` `ProtocolMessage`s must be temporarily stored in the
+    `OBJECT_SYNC` `ProtocolMessages` must be temporarily stored in the
     internal `SyncObjectsPool` list
   - `(RTO5c)` When the objects sync has completed, the client library
     must perform the following actions in order:
@@ -143,9 +145,13 @@ the latest state.
       which `objectId`s were not received during the sync sequence
       - `(RTO5c2a)` The object with ID `root` must not be removed from
         `ObjectsPool`, as per [RTO3b](#RTO3b)
+    - `(RTO5c6)` `ObjectMessages` stored in the
+      `BufferedObjectOperations` list are applied as described in
+      [RTO9](#RTO9)
     - `(RTO5c3)` Clear any stored sync sequence identifiers and cursor
       values
     - `(RTO5c4)` The `SyncObjectsPool` must be cleared
+    - `(RTO5c5)` The `BufferedObjectOperations` list must be cleared
 - `(RTO6)` Certain object operations may require creating a zero-value
   object if one does not already exist in the internal `ObjectsPool` for
   the given `objectId`. This can be done as follows:
@@ -160,11 +166,100 @@ the latest state.
       `LiveMap` per [RTLM4](#RTLM4) in the `ObjectsPool`
     - `(RTO6b3)` If the parsed type is `counter`, create a zero-value
       `LiveCounter` per [RTLC4](#RTLC4) in the `ObjectsPool`
+- `(RTO7)` The client library may receive `OBJECT` `ProtocolMessages` in
+  realtime over the channel concurrently with `OBJECT_SYNC`
+  `ProtocolMessages` during the object sync sequence ([RTO5](#RTO5)).
+  Some of the incoming `OBJECT` messages may have already been applied
+  to the objects described in the sync sequence, while others may not.
+  Therefore, the client must buffer `OBJECT` messages during the sync
+  sequence so that it can determine which of them should be applied to
+  the objects once the sync is complete. See [RTO8](#RTO8)
+  - `(RTO7a)` An internal `BufferedObjectOperations` should be used to
+    store the buffered `ObjectMessages`, as described in
+    [RTO8a](#RTO8a). `BufferedObjectOperations` is an array of
+    `ObjectMessage` instances
+    - `(RTO7a1)` This array is empty upon `RealtimeObjects`
+      initialization
+- `(RTO8)` When the library receives a `ProtocolMessage` with an action
+  of `OBJECT`, each member of the `ProtocolMessage.state` array (decoded
+  into `ObjectMessage` objects) is passed to the `RealtimeObjects`
+  instance per [RTL1](../features#RTL1). Each `ObjectMessage` from
+  `OBJECT` `ProtocolMessage` (also referred to as an `OBJECT` message)
+  describes an operation to be applied to an object on a channel and
+  must be handled as follows:
+  - `(RTO8a)` If an object sync sequence is currently in progress, add
+    the `ObjectMessages` to the internal `BufferedObjectOperations`
+    array
+  - `(RTO8b)` Otherwise, apply the `ObjectMessages` as described in
+    [RTO9](#RTO9)
+- `(RTO9)` `OBJECT` messages can be applied to `RealtimeObjects` in the
+  following way:
+  - `(RTO9a)` For each `ObjectMessage` in the provided list:
+    - `(RTO9a1)` If `ObjectMessage.operation` is null or omitted, log a
+      warning indicating that an unsupported object operation message
+      has been received, and discard the current `ObjectMessage` without
+      taking any action
+    - `(RTO9a2)` The `ObjectMessage.operation.action` field (see
+      [`ObjectOperationAction`](../features#OOP2)) determines the type
+      of operation to apply:
+      - `(RTO9a2a)` If `ObjectMessage.operation.action` is one of the
+        following: `MAP_CREATE`, `MAP_SET`, `MAP_REMOVE`,
+        `COUNTER_CREATE`, `COUNTER_INC`, or `OBJECT_DELETE`, then:
+        - `(RTO9a2a1)` If it does not already exist, create a zero-value
+          `LiveObject` in the internal `ObjectsPool` per [RTO6](#RTO6)
+          using the `objectId` from `ObjectMessage.operation.objectId`
+        - `(RTO9a2a2)` Get the `LiveObject` instance from the internal
+          `ObjectsPool` using the `objectId` from
+          `ObjectMessage.operation.objectId`
+        - `(RTO9a2a3)` Apply the `ObjectMessage.operation` to the
+          `LiveObject`; see [RTLC7](#RTLC7), [RTLM15](#RTLM15)
+      - `(RTO9a2b)` Otherwise, log a warning that an object operation
+        message with an unsupported action has been received, and
+        discard the current `ObjectMessage` without taking any action
 
 ### LiveObject
 
 - `(RTLO1)` The `LiveObject` represents the common interface and
   includes shared functionality for concrete object types
+- `(RTLO2)` The client library may choose to implement `LiveObject` as
+  an abstract class
+- `(RTLO3)` `LiveObject` properties:
+  - `(RTLO3a)` protected `objectId` string - an Object ID for this
+    object
+    - `(RTLO3a1)` Must be provided and set in the constructor
+  - `(RTLO3b)` protected `siteTimeserials` `Dict<String, String>` - a
+    map of [serials](../features#OM2h) keyed by
+    [siteCode](../features#OM2i), representing the last operations
+    applied to this object
+    - `(RTLO3b1)` Set to an empty map when the `LiveObject` is
+      initialized, so that any future operation can be applied to this
+      object
+  - `(RTLO3c)` protected `createOperationIsMerged` boolean - a flag
+    indicating whether the corresponding `MAP_CREATE` or
+    `COUNTER_CREATE` operation has been applied to this `LiveObject`
+    instance
+    - `(RTLO3c1)` Set to `false` when the `LiveObject` is initialized
+- `(RTLO4)` `LiveObject` methods:
+  - `(RTLO4a)` protected `canApplyOperation` - a convenience method used
+    to determine whether the `ObjectMessage.operation` should be applied
+    to this object based on a serial value:
+    - `(RTLO4a1)` Expects the following arguments:
+      - `(RTLO4a1a)` `ObjectMessage`
+    - `(RTLO4a2)` Returns a boolean indicating whether the operation
+      should be applied to this object
+    - `(RTLO4a3)` Both `ObjectMessage.serial` and
+      `ObjectMessage.siteCode` must be non-empty strings. Otherwise, log
+      a warning that the object operation message has invalid serial
+      values. The client library must not apply this operation to the
+      object
+    - `(RTLO4a4)` Get the `siteSerial` value stored for this
+      `LiveObject` in the `siteTimeserials` map using the key
+      `ObjectMessage.siteCode`
+    - `(RTLO4a5)` If the `siteSerial` for this `LiveObject` is null or
+      an empty string, return true
+    - `(RTLO4a6)` If the `siteSerial` for this `LiveObject` is not an
+      empty string, return true if `ObjectMessage.serial` is greater
+      than `siteSerial` when compared lexicographically
 
 ### LiveCounter
 
@@ -188,11 +283,60 @@ the latest state.
   - `(RTLC6b)` Set the private flag `createOperationIsMerged` to `false`
   - `(RTLC6c)` Set `data` to the value of `ObjectState.counter.count`,
     or to 0 if it does not exist
-  - `(RTLC6d)` If `ObjectState.createOp` is present:
-    - `(RTLC6d1)` Add `ObjectState.createOp.counter.count` to `data`, if
-      it exists
-    - `(RTLC6d2)` Set the private flag `createOperationIsMerged` to
-      `true`
+  - `(RTLC6d)` If `ObjectState.createOp` is present, merge the initial
+    value into the `LiveCounter` as described in [RTLC10](#RTLC10),
+    passing in the `ObjectState.createOp` instance
+    - `(RTLC6d1)` This clause has been replaced by [RTLC10a](#RTLC10a)
+    - `(RTLC6d2)` This clause has been replaced by [RTLC10b](#RTLC10b)
+- `(RTLC7)` An `ObjectOperation` from `ObjectMessage.operation` can be
+  applied to a `LiveCounter` in the following way:
+  - `(RTLC7a)` A client library may choose to implement this logic as a
+    convenience method named `applyOperation`, which accepts an
+    `ObjectMessage` instance with an existing `ObjectMessage.operation`
+    object, with `ObjectMessage.operation.objectId` matching the Object
+    ID of this `LiveCounter`. This `ObjectMessage` represents the
+    operation to be applied to this `LiveCounter`
+  - `(RTLC7b)` If `ObjectMessage.operation` cannot be applied based on
+    the result of [`LiveObject.canApplyOperation`](#RTLO4a), log a debug
+    or trace message indicating that the operation cannot be applied
+    because its serial value is not newer than the object’s, and discard
+    the `ObjectMessage` without taking any action
+  - `(RTLC7c)` Set the entry in the private `siteTimeserials` map at the
+    key `ObjectMessage.siteCode` to equal `ObjectMessage.serial`
+  - `(RTLC7d)` The `ObjectMessage.operation.action` field (see
+    [`ObjectOperationAction`](../features#OOP2)) determines the type of
+    operation to apply:
+    - `(RTLC7d1)` If `ObjectMessage.operation.action` is set to
+      `COUNTER_CREATE`, apply the operation as described in
+      [RTLC8](#RTLC8), passing in `ObjectMessage.operation`
+    - `(RTLC7d2)` If `ObjectMessage.operation.action` is set to
+      `COUNTER_INC`, apply the operation as described in
+      [RTLC9](#RTLC9), passing in `ObjectMessage.operation.counterOp`
+    - `(RTLC7d3)` Otherwise, log a warning that an object operation
+      message with an unsupported action has been received, and discard
+      the current `ObjectMessage` without taking any action
+- `(RTLC8)` A `COUNTER_CREATE` operation can be applied to a
+  `LiveCounter` in the following way:
+  - `(RTLC8a)` Expects the following arguments:
+    - `(RTLC8a1)` `ObjectOperation`
+  - `(RTLC8b)` If the private flag `createOperationIsMerged` is `true`,
+    log a debug or trace message indicating that the operation will not
+    be applied because a `COUNTER_CREATE` operation has already been
+    applied to this `LiveCounter`, and discard the operation without
+    taking any further action
+  - `(RTLC8c)` Otherwise merge the initial value into the `LiveCounter`
+    as described in [RTLC10](#RTLC10), passing in the `ObjectOperation`
+    instance
+- `(RTLC9)` A `COUNTER_INC` operation can be applied to a `LiveCounter`
+  in the following way:
+  - `(RTLC9a)` Expects the following arguments:
+    - `(RTLC9a1)` `ObjectsCounterOp`
+  - `(RTLC9b)` Add `ObjectsCounterOp.amount` to `data`, if it exists
+- `(RTLC10)` The initial value from `ObjectOperation.counter` can be
+  merged into this `LiveCounter` in the following way:
+  - `(RTLC10a)` Add `ObjectOperation.counter.count` to `data`, if it
+    exists
+  - `(RTLC10b)` Set the private flag `createOperationIsMerged` to `true`
 
 ### LiveMap
 
@@ -289,22 +433,65 @@ the latest state.
   - `(RTLM6b)` Set the private flag `createOperationIsMerged` to `false`
   - `(RTLM6c)` Set `data` to `ObjectState.map.entries`, or to an empty
     map if it does not exist
-  - `(RTLM6d)` If `ObjectState.createOp` is present:
-    - `(RTLM6d1)` For each key–@ObjectsMapEntry@ pair in
-      `ObjectState.createOp.map.entries`:
-      - `(RTLM6d1a)` If `ObjectsMapEntry.tombstone` is `false` or
-        omitted, apply the `MAP_SET` operation to the current key as
-        described in [RTLM7](#RTLM7), passing in `ObjectsMapEntry.data`
-        and the current key as `ObjectsMapOp`, and
-        `ObjectsMapEntry.timeserial` as `serial`
-      - `(RTLM6d1b)` If `ObjectsMapEntry.tombstone` is `true`, apply the
-        `MAP_REMOVE` operation to the current key as described in
-        [RTLM8](#RTLM8), passing in the current key as `ObjectsMapOp`,
-        and `ObjectsMapEntry.timeserial` as `serial`
-    - `(RTLM6d2)` Set the private flag `createOperationIsMerged` to
-      `true`
-- `(RTLM7)` `MAP_SET` operation for a key can be applied to a `LiveMap`
-  in the following way:
+  - `(RTLM6d)` If `ObjectState.createOp` is present, merge the initial
+    value into the `LiveMap` as described in [RTLM17](#RTLM17), passing
+    in the `ObjectState.createOp` instance
+    - `(RTLM6d1)` This clause has been replaced by [RTLM17a](#RTLM17a)
+      - `(RTLM6d1a)` This clause has been replaced by
+        [RTLM17a1](#RTLM17a1)
+      - `(RTLM6d1b)` This clause has been replaced by
+        [RTLM17a2](#RTLM17a2)
+    - `(RTLM6d2)` This clause has been replaced by [RTLM17b](#RTLM17b)
+- `(RTLM15)` An `ObjectOperation` from `ObjectMessage.operation` can be
+  applied to a `LiveMap` in the following way:
+  - `(RTLM15a)` A client library may choose to implement this logic as a
+    convenience method named `applyOperation`, which accepts an
+    `ObjectMessage` instance with an existing `ObjectMessage.operation`
+    object, with `ObjectMessage.operation.objectId` matching the Object
+    ID of this `LiveMap`. This `ObjectMessage` represents the operation
+    to be applied to this `LiveMap`
+  - `(RTLM15b)` If `ObjectMessage.operation` cannot be applied based on
+    the result of [`LiveObject.canApplyOperation`](#RTLO4a), log a debug
+    or trace message indicating that the operation cannot be applied
+    because its serial value is not newer than the object’s, and discard
+    the `ObjectMessage` without taking any action
+  - `(RTLM15c)` Set the entry in the private `siteTimeserials` map at
+    the key `ObjectMessage.siteCode` to equal `ObjectMessage.serial`
+  - `(RTLM15d)` The `ObjectMessage.operation.action` field (see
+    [`ObjectOperationAction`](../features#OOP2)) determines the type of
+    operation to apply:
+    - `(RTLM15d1)` If `ObjectMessage.operation.action` is set to
+      `MAP_CREATE`, apply the operation as described in
+      [RTLM16](#RTLM16), passing in `ObjectMessage.operation`
+    - `(RTLM15d2)` If `ObjectMessage.operation.action` is set to
+      `MAP_SET`, apply the operation as described in [RTLM7](#RTLM7),
+      passing in `ObjectMessage.operation.mapOp` and
+      `ObjectMessage.serial`
+    - `(RTLM15d3)` If `ObjectMessage.operation.action` is set to
+      `MAP_REMOVE`, apply the operation as described in [RTLM8](#RTLM8),
+      passing in `ObjectMessage.operation.mapOp` and
+      `ObjectMessage.serial`
+    - `(RTLM15d4)` Otherwise, log a warning that an object operation
+      message with an unsupported action has been received, and discard
+      the current `ObjectMessage` without taking any action
+- `(RTLM16)` A `MAP_CREATE` operation can be applied to a `LiveMap` in
+  the following way:
+  - `(RTLM16a)` Expects the following argument:
+    - `(RTLM16a1)` `ObjectOperation`
+  - `(RTLM16b)` If the private flag `createOperationIsMerged` is `true`,
+    log a debug or trace message indicating that the operation will not
+    be applied because a `MAP_CREATE` operation has already been applied
+    to this `LiveMap`, and discard the operation without taking any
+    further action
+  - `(RTLM16c)` If the private `semantics` field does not match
+    `ObjectOperation.map.semantics`, log a warning that the operation
+    cannot be applied due to mismatched semantics, and discard the
+    operation without taking any further action
+  - `(RTLM16d)` Otherwise merge the initial value into the `LiveMap` as
+    described in [RTLM17](#RTLM17), passing in the `ObjectOperation`
+    instance
+- `(RTLM7)` A `MAP_SET` operation for a key can be applied to a
+  `LiveMap` in the following way:
   - `(RTLM7d)` Expects the following arguments:
     - `(RTLM7d1)` `ObjectsMapOp`
     - `(RTLM7d2)` `serial` string - operation’s serial value
@@ -371,3 +558,17 @@ the latest state.
   - `(RTLM9e)` If both serials exist and are not empty strings, compare
     them lexicographically and allow operation to be applied only if the
     operation’s serial is greater than the entry’s serial
+- `(RTLM17)` The initial value from `ObjectOperation.map` can be merged
+  into this `LiveMap` in the following way:
+  - `(RTLM17a)` For each key–@ObjectsMapEntry@ pair in
+    `ObjectOperation.map.entries`:
+    - `(RTLM17a1)` If `ObjectsMapEntry.tombstone` is `false` or omitted,
+      apply the `MAP_SET` operation to the current key as described in
+      [RTLM7](#RTLM7), passing in `ObjectsMapEntry.data` and the current
+      key as `ObjectsMapOp`, and `ObjectsMapEntry.timeserial` as
+      `serial`
+    - `(RTLM17a2)` If `ObjectsMapEntry.tombstone` is `true`, apply the
+      `MAP_REMOVE` operation to the current key as described in
+      [RTLM8](#RTLM8), passing in the current key as `ObjectsMapOp`, and
+      `ObjectsMapEntry.timeserial` as `serial`
+  - `(RTLM17b)` Set the private flag `createOperationIsMerged` to `true`