You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Simplify job reservation: no locking, rely on make() transaction
- Remove SELECT FOR UPDATE locking from job reservation
- Conflicts (rare) resolved by make() transaction's duplicate key error
- Second worker catches error and moves to next job
- Simpler code, better performance on high-traffic jobs table
The jobs table is created with the appropriate primary key structure matching the target table's foreign-key-derived attributes.
437
437
438
-
### Race Condition Handling
438
+
### Conflict Resolution
439
439
440
-
Job reservation uses database-level locking to prevent race conditions:
440
+
Job reservation does **not** use transaction-level locking for simplicity and performance. Instead, conflicts are resolved at the `make()` transaction level:
441
441
442
-
```sql
443
-
-- Reserve a job atomically
444
-
START TRANSACTION;
445
-
SELECT*FROM`_my_table__jobs`
446
-
WHERE status ='pending'
447
-
AND scheduled_time <= NOW()
448
-
ORDER BY priority DESC, scheduled_time ASC
449
-
LIMIT1
450
-
FOR UPDATE SKIP LOCKED;
451
-
452
-
-- If row found, update it
442
+
```python
443
+
# Simple reservation (no locking)
453
444
UPDATE`_my_table__jobs`
454
445
SET status ='reserved',
455
446
reserved_time = NOW(),
456
447
user = CURRENT_USER(),
457
448
host =@@hostname,
458
449
pid = CONNECTION_ID()
459
-
WHERE<primary_key_match>;
460
-
461
-
COMMIT;
450
+
WHERE status ='pending'
451
+
AND scheduled_time <= NOW()
452
+
ORDERBY priority DESC, scheduled_time ASC
453
+
LIMIT1;
462
454
```
463
455
456
+
**Conflict scenario** (rare):
457
+
1. Two workers reserve the same job nearly simultaneously
458
+
2. Both run `make()` for the same key
459
+
3. First worker's `make()` transaction commits, inserting the result
460
+
4. Second worker's `make()` transaction fails with duplicate key error
461
+
5. Second worker catches the error and moves to the next job
462
+
463
+
**Why this is acceptable**:
464
+
- Conflicts are rare in practice (requires near-simultaneous reservation)
465
+
- The `make()` transaction already guarantees data integrity
466
+
- Duplicate key error is a clean, expected signal
467
+
- Avoids locking overhead on the high-traffic jobs table
468
+
- Wasted computation is minimal compared to locking complexity
469
+
464
470
### Stale Reserved Job Detection
465
471
466
472
Reserved jobs that have been running too long may indicate crashed workers:
0 commit comments