|
1 | | -# sync |
2 | | -Offline-first sync engine (WAL, outbox, retries). |
| 1 | +# Vix Sync |
| 2 | + |
| 3 | +**Durable. Offline-first. Deterministic.** |
| 4 | + |
| 5 | +Vix Sync is the core synchronization engine of Vix.cpp. |
| 6 | + |
| 7 | +It guarantees that every operation: |
| 8 | +- is persisted before being sent |
| 9 | +- is never lost |
| 10 | +- is retried deterministically |
| 11 | +- converges to a consistent state |
| 12 | + |
| 13 | +--- |
| 14 | + |
| 15 | +## Why Vix Sync exists |
| 16 | + |
| 17 | +Real-world systems are not reliable: |
| 18 | +- network drops |
| 19 | +- servers restart |
| 20 | +- requests fail halfway |
| 21 | + |
| 22 | +Traditional systems: |
| 23 | +- lose data |
| 24 | +- duplicate requests |
| 25 | +- become inconsistent |
| 26 | + |
| 27 | +Vix Sync solves this with a simple rule: |
| 28 | + |
| 29 | +> Write locally. Persist first. Sync later. |
| 30 | +
|
| 31 | +--- |
| 32 | + |
| 33 | +## Core principles |
| 34 | + |
| 35 | +1. **Durability first** |
| 36 | + - Every operation is stored before any network call |
| 37 | + |
| 38 | +2. **Deterministic retries** |
| 39 | + - Retry logic is reproducible and replayable |
| 40 | + |
| 41 | +3. **Offline-first** |
| 42 | + - The system works without network |
| 43 | + |
| 44 | +4. **Safe convergence** |
| 45 | + - Eventually, all nodes reach a consistent state |
| 46 | + |
| 47 | +--- |
| 48 | + |
| 49 | +## Architecture |
| 50 | + |
| 51 | +Vix Sync is composed of small deterministic building blocks: |
| 52 | + |
| 53 | +```text |
| 54 | +Local Write |
| 55 | +↓ |
| 56 | +WAL (Write-Ahead Log) |
| 57 | +↓ |
| 58 | +Outbox (durable queue) |
| 59 | +↓ |
| 60 | +SyncWorker |
| 61 | +↓ |
| 62 | +Transport (HTTP / P2P / custom) |
| 63 | +↓ |
| 64 | +Done / Retry / Failed |
| 65 | +``` |
| 66 | + |
| 67 | +--- |
| 68 | + |
| 69 | +## Components |
| 70 | + |
| 71 | +### 1. Operation |
| 72 | + |
| 73 | +A durable unit of work. |
| 74 | + |
| 75 | +```cpp |
| 76 | +struct Operation { |
| 77 | + std::string id; |
| 78 | + std::string kind; |
| 79 | + std::string target; |
| 80 | + std::string payload; |
| 81 | +}; |
| 82 | +``` |
| 83 | +
|
| 84 | +### 2. WAL (Write-Ahead Log) |
| 85 | +
|
| 86 | +Append-only log: |
| 87 | +
|
| 88 | +- crash-safe |
| 89 | +- replayable |
| 90 | +- ordered |
| 91 | +
|
| 92 | +```cpp |
| 93 | +Wal wal({ .file_path = "./.vix/wal.log" }); |
| 94 | +
|
| 95 | +wal.append(record); |
| 96 | +wal.replay(0, handler); |
| 97 | +``` |
| 98 | + |
| 99 | +### 3. Outbox |
| 100 | + |
| 101 | +Durable operation queue: |
| 102 | + |
| 103 | +- enqueue |
| 104 | +- claim |
| 105 | +- complete |
| 106 | +- retry |
| 107 | + |
| 108 | +```cpp |
| 109 | +auto id = outbox->enqueue(op, now); |
| 110 | +outbox->claim(id, now); |
| 111 | +outbox->complete(id, now); |
| 112 | +``` |
| 113 | +
|
| 114 | +### 4. RetryPolicy |
| 115 | +
|
| 116 | +Deterministic exponential backoff: |
| 117 | +
|
| 118 | +```cpp |
| 119 | +RetryPolicy retry; |
| 120 | +retry.base_delay_ms = 500; |
| 121 | +retry.factor = 2.0; |
| 122 | +retry.max_attempts = 8; |
| 123 | +``` |
| 124 | + |
| 125 | +### 5. SyncWorker |
| 126 | + |
| 127 | +Processes operations: |
| 128 | + |
| 129 | +- fetch ready ops |
| 130 | +- send via transport |
| 131 | +- mark done / fail |
| 132 | + |
| 133 | +### 6. SyncEngine |
| 134 | + |
| 135 | +Orchestrates workers: |
| 136 | + |
| 137 | +```cpp |
| 138 | +SyncEngine engine(cfg, outbox, probe, transport); |
| 139 | +engine.tick(now); |
| 140 | +``` |
| 141 | +
|
| 142 | +--- |
| 143 | +
|
| 144 | +## Examples |
| 145 | +
|
| 146 | +Run examples with: |
| 147 | +
|
| 148 | +```bash |
| 149 | +vix run examples/<file>.cpp |
| 150 | +``` |
| 151 | + |
| 152 | +### Basic |
| 153 | +```bash |
| 154 | +vix run examples/01_outbox_basic.cpp |
| 155 | +``` |
| 156 | + |
| 157 | +### Retry & failure |
| 158 | +```bash |
| 159 | +vix run examples/02_outbox_retry_and_failure.cpp |
| 160 | +``` |
| 161 | + |
| 162 | +### WAL |
| 163 | +```bash |
| 164 | +vix run examples/03_wal_append_and_replay.cpp |
| 165 | +``` |
| 166 | + |
| 167 | +### Worker |
| 168 | +```bash |
| 169 | +vix run examples/04_sync_worker_success.cpp |
| 170 | +``` |
| 171 | + |
| 172 | +### Permanent failure |
| 173 | +```bash |
| 174 | +vix run examples/05_sync_worker_permanent_failure.cpp |
| 175 | +``` |
| 176 | + |
| 177 | +### Engine requeue |
| 178 | +```bash |
| 179 | +vix run examples/06_sync_engine_requeue_inflight.cpp |
| 180 | +``` |
| 181 | + |
| 182 | +### End-to-end (local-first) |
| 183 | +```bash |
| 184 | +vix run examples/07_local_write_wal_outbox_engine.cpp |
| 185 | +``` |
| 186 | + |
| 187 | +### Offline to recovery |
| 188 | +```bash |
| 189 | +vix run examples/08_local_write_offline_then_recover.cpp |
| 190 | +``` |
| 191 | + |
| 192 | +--- |
| 193 | + |
| 194 | +## Key invariant |
| 195 | + |
| 196 | +If the process crashes at any point, the system can recover fully. |
| 197 | + |
| 198 | +Because: |
| 199 | + |
| 200 | +- WAL stores intent |
| 201 | +- Outbox stores state |
| 202 | +- Retry is deterministic |
| 203 | + |
| 204 | +--- |
| 205 | + |
| 206 | +## When to use Vix Sync |
| 207 | + |
| 208 | +Use it when you need: |
| 209 | + |
| 210 | +- offline-first systems |
| 211 | +- reliable message delivery |
| 212 | +- distributed state convergence |
| 213 | +- retry-safe APIs |
| 214 | +- edge or unstable network environments |
| 215 | + |
| 216 | +--- |
| 217 | + |
| 218 | +## What Vix Sync is NOT |
| 219 | + |
| 220 | +- not a message broker |
| 221 | +- not a queue system like Kafka |
| 222 | +- not a distributed database |
| 223 | + |
| 224 | +It is: |
| 225 | + |
| 226 | +> a durable sync layer for real-world unreliable systems |
| 227 | +
|
| 228 | +--- |
| 229 | + |
| 230 | +## Future |
| 231 | + |
| 232 | +- P2P transport |
| 233 | +- conflict resolution |
| 234 | +- multi-device sync |
| 235 | +- edge replication |
| 236 | + |
| 237 | +--- |
| 238 | + |
| 239 | +## License |
| 240 | + |
| 241 | +MIT License |
| 242 | +© Gaspard Kirira |
| 243 | + |
0 commit comments