Skip to content

Commit 48f818d

Browse files
committed
fix(data-drains): cadence buffer + clear stale error on success
- Dispatcher: subtract a 5-minute buffer from the cadence cutoff. Without this, a drain that finishes a few minutes after the hourly tick (lastRunAt = 10:05) won't satisfy lastRunAt < now - 1h at the next tick, causing "hourly" drains to run every ~2 hours. - Service: clear `error` to null on the success update so a row previously reaped as orphaned doesn't render as success-with-error.
1 parent 59410fc commit 48f818d

2 files changed

Lines changed: 13 additions & 2 deletions

File tree

apps/sim/lib/data-drains/dispatcher.ts

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,16 @@ const logger = createLogger('DataDrainsDispatcher')
1212
const HOUR_MS = 60 * 60 * 1000
1313
const DAY_MS = 24 * HOUR_MS
1414

15+
/**
16+
* Cron fires hourly. Without a buffer, a drain that finishes a few minutes
17+
* after the tick (lastRunAt = 10:05) won't satisfy `lastRunAt < now - cadence`
18+
* at the next tick (10:05 < 10:00 is false), so an "hourly" drain effectively
19+
* runs every two hours. Subtracting a small buffer from the cadence absorbs
20+
* normal run duration plus cron jitter without allowing back-to-back runs
21+
* within the same tick.
22+
*/
23+
const CADENCE_BUFFER_MS = 5 * 60 * 1000
24+
1525
/**
1626
* Maximum wall-clock duration any single drain run is allowed before its
1727
* `data_drain_runs` row is considered orphaned. Runs that exceed this are
@@ -59,8 +69,8 @@ export async function dispatchDueDrains(now: Date = new Date()): Promise<{
5969
}> {
6070
const { reaped } = await reapOrphanedRuns(now)
6171

62-
const hourlyCutoff = new Date(now.getTime() - HOUR_MS)
63-
const dailyCutoff = new Date(now.getTime() - DAY_MS)
72+
const hourlyCutoff = new Date(now.getTime() - HOUR_MS + CADENCE_BUFFER_MS)
73+
const dailyCutoff = new Date(now.getTime() - DAY_MS + CADENCE_BUFFER_MS)
6474

6575
const duePredicate = and(
6676
eq(dataDrains.enabled, true),

apps/sim/lib/data-drains/service.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -144,6 +144,7 @@ export async function runDrain(
144144
bytesWritten,
145145
cursorAfter: cursor,
146146
locators,
147+
error: null,
147148
})
148149
.where(eq(dataDrainRuns.id, runId))
149150
})

0 commit comments

Comments
 (0)