diff --git a/.specs/kiloclaw-billing-lifecycle.md b/.specs/kiloclaw-billing-lifecycle.md index 0409c60278..26ea959154 100644 --- a/.specs/kiloclaw-billing-lifecycle.md +++ b/.specs/kiloclaw-billing-lifecycle.md @@ -151,6 +151,11 @@ downstream enforcement can proceed. 9. Processing MUST preserve the period-scoped idempotency behavior defined in `.specs/kiloclaw-billing.md` so duplicate delivery cannot double-charge or double-advance a subscription. +10. Every successful renewal or duplicate-boundary reconciliation that + leaves or transitions the subscription to active MUST clear any prior + destruction deadline in the successful renewal transaction. This + invalidation MUST NOT wait for instance start, readiness, or auto-resume + completion, and MUST NOT clear the suspension timestamp. ### Shared Credit Balance Safety @@ -240,6 +245,11 @@ downstream enforcement can proceed. 6. If credit-renewal backlog or retry age creates credible risk of false suspension or destruction, the system SHOULD add a stronger protection mechanism before continuing rollout. +7. Immediately before requesting personal instance destruction, downstream + enforcement MUST re-read and revalidate the current subscription and + instance. Destruction MUST be skipped unless the row remains current, + suspended, destruction-eligible, past its deadline, attached to the same + live personal instance, and the instance is not already destroyed. ### Observability and Operator Control diff --git a/.specs/kiloclaw-billing.md b/.specs/kiloclaw-billing.md index d88e401b32..3ee2c9a8a3 100644 --- a/.specs/kiloclaw-billing.md +++ b/.specs/kiloclaw-billing.md @@ -933,8 +933,11 @@ rows renew. operations so that a retry can re-attempt the deduction without the idempotency key blocking it. 4. If the deduction insert returns zero affected rows (duplicate key - from a prior committed transaction), the subscription update - within the same transaction is a no-op (same values). The system + from a prior committed transaction), the system MUST reconcile the + subscription advancement when the same renewal boundary remains + current. Reconciliation MUST apply the same active-renewal destruction- + deadline invalidation and suspended-recovery side effects as a newly + inserted deduction. If the boundary has already advanced, the system MUST skip further processing for that row. 5. If the subscription has cancel-at-period-end set, the sweep MUST skip the deduction, set the subscription status to canceled, and @@ -945,10 +948,12 @@ rows renew. 6. When the effective balance (as defined in Credit Enrollment rule 4) is sufficient for the subscription's price-version renewal amount and the deduction succeeds (one affected row), the system MUST - atomically record the deduction as credit spend (see Definitions) - and advance the subscription's billing period - (current-period-start, current-period-end, credit-renewal-timestamp) - within the same transaction. After the transaction commits, the + atomically record the deduction as credit spend (see Definitions), + advance the subscription's billing period + (current-period-start, current-period-end, credit-renewal-timestamp), + and clear any prior destruction deadline within the same transaction. + Clearing the destruction deadline MUST NOT clear the suspension + timestamp or auto-resume retry state. After the transaction commits, the system MUST trigger a bonus credit evaluation as described in Credit Enrollment rule 6. The user's credit balance MAY be temporarily negative between the deduction and the bonus award. If the bonus @@ -967,9 +972,11 @@ rows renew. the user so that future failures can re-trigger the notification. 10. When the deduction succeeds, the subscription was past-due, and the suspension timestamp is non-null (suspended recovery), the - system MUST call the auto-resume procedure to restart the instance, - clear the suspension-cycle email log entries (including the - credit-renewal-failed entry), and clear the suspension columns. + system MUST call the auto-resume procedure to restart the instance. + The renewal transaction has already invalidated the prior destruction + deadline; suspension-cycle email log entries (including the credit- + renewal-failed entry), the suspension timestamp, and auto-resume retry + state MUST remain until readiness completion. 11. When the effective balance (as defined in Credit Enrollment rule 4) is insufficient, the system MUST first check whether the user has auto top-up enabled and whether a top-up has @@ -1127,6 +1134,11 @@ rows renew. hard-expiry enforcement, the system MUST re-evaluate parent organization entitlement. If entitlement has returned, the system MUST run Organization Entitlement Recovery instead of sending the warning. +3. Immediately before sending a personal destruction warning, the system + MUST re-read the current subscription and instance state and skip the + warning if the row is no longer current and destruction-eligible, the + deadline no longer falls within the warning window, or the instance is + no longer the same live personal instance. ### Instance Destruction @@ -1153,6 +1165,12 @@ rows renew. MUST emit telemetry identifying pending provider resources and the latest provider error, if any. A provider 404 for a resource counts as confirmed deletion of that resource. +9. Immediately before requesting destruction of a personal instance, the + system MUST re-read the current subscription and instance state. It MUST + skip destruction unless the same current subscription still has a + destruction-eligible status, remains suspended, has an elapsed deadline, + still references the same live personal instance, and that instance is + not already destroyed. ### Past-Due Payment Enforcement @@ -1213,16 +1231,21 @@ rows renew. status-change events MUST NOT trigger auto-resume for hybrid rows (see Hybrid Subscription Ownership rule 2). 2. If the instance start attempt fails, the system MUST log the failure - and MUST NOT clear the suspension timestamp or destruction deadline. + and MUST NOT clear the suspension timestamp or auto-resume retry state. Leaving these fields intact allows the background job (Billing - Lifecycle Background Job rule 5) to detect the incomplete - auto-resume and retry on the next sweep. -3. The system MUST clear the suspension timestamp and destruction - deadline only after a successful instance start (or when no instance - exists to restart). -4. The system MUST clear email log entries for suspension, destruction, + Lifecycle Background Job rule 5) to detect the incomplete auto-resume + and retry on the next sweep. A successful credit renewal MUST already + have invalidated the prior destruction deadline before the start attempt, + and start failure MUST NOT restore it. +3. The system MUST clear the suspension timestamp and auto-resume retry + state only after a successful instance start (or when no instance + exists to restart). Credit-renewal destruction-deadline invalidation + MUST NOT wait for instance readiness or auto-resume completion. +4. After successful instance start (or when no instance exists to restart), + the system MUST clear email log entries for suspension, destruction, and credit-renewal-failed notifications so they can fire again in a - future suspension cycle. + future suspension cycle. Deadline invalidation alone MUST NOT reset + those entries while the suspension timestamp remains non-null. 5. The system MUST NOT clear email log entries for trial or earlybird warning notifications, as those are one-time events. diff --git a/apps/web/src/lib/kiloclaw/instance-lifecycle.test.ts b/apps/web/src/lib/kiloclaw/instance-lifecycle.test.ts index 93c5a418c0..a79ba1aa90 100644 --- a/apps/web/src/lib/kiloclaw/instance-lifecycle.test.ts +++ b/apps/web/src/lib/kiloclaw/instance-lifecycle.test.ts @@ -25,6 +25,28 @@ const txInsertValues: Array> = []; const deleteWhereCalls: unknown[] = []; const startAsyncMock = jest.fn(); +function sqlParamValues(condition: unknown): unknown[] { + const values: unknown[] = []; + const visited = new WeakSet(); + + const visit = (value: unknown) => { + if (!value || typeof value !== 'object' || visited.has(value)) return; + visited.add(value); + + if (value.constructor.name === 'Param' && 'value' in value) { + values.push(value.value); + return; + } + + for (const child of Object.values(value)) { + visit(child); + } + }; + + visit(condition); + return values; +} + function createSelectResult(rows: T[]) { const promise = Promise.resolve(rows); const result: { @@ -208,7 +230,7 @@ describe('instance lifecycle async resume', () => { ); }); - it('completes async auto-resume for an active subscription and clears retry state', async () => { + it('completes async auto-resume and resets warning dedupe for a later suspension lifecycle', async () => { const instanceId = '11111111-1111-4111-8111-111111111111'; const sandboxId = 'ki_11111111111141118111111111111111'; selectResultsQueue.push( @@ -243,6 +265,16 @@ describe('instance lifecycle async resume', () => { expect(result).toEqual({ instanceId, resumeCompleted: true }); expect(mockDb.transaction).toHaveBeenCalledTimes(1); expect(deleteWhereCalls).toHaveLength(1); + expect(sqlParamValues(deleteWhereCalls[0])).toEqual( + expect.arrayContaining([ + 'claw_suspended_trial', + 'claw_suspended_subscription', + 'claw_suspended_payment', + 'claw_destruction_warning', + 'claw_instance_destroyed', + 'claw_credit_renewal_failed', + ]) + ); expect(txUpdateSetCalls).toHaveLength(1); expect(txUpdateSetCalls[0]).toEqual({ suspended_at: null, @@ -257,6 +289,17 @@ describe('instance lifecycle async resume', () => { actor_id: 'web-instance-lifecycle', action: 'reactivated', reason: 'auto_resume_completed', + before_state: expect.objectContaining({ + suspended_at: '2026-04-07T20:00:00.000Z', + destruction_deadline: '2026-04-14T20:00:00.000Z', + }), + after_state: expect.objectContaining({ + suspended_at: null, + destruction_deadline: null, + auto_resume_requested_at: null, + auto_resume_retry_after: null, + auto_resume_attempt_count: 0, + }), }) ); }); diff --git a/services/kiloclaw-billing/src/lifecycle.test.ts b/services/kiloclaw-billing/src/lifecycle.test.ts index fde40b9157..6b751401c9 100644 --- a/services/kiloclaw-billing/src/lifecycle.test.ts +++ b/services/kiloclaw-billing/src/lifecycle.test.ts @@ -383,6 +383,31 @@ function organizationDestructionWarningRow(overrides: Partial> = {}) { + return { + id: '55555555-5555-4555-8555-555555555555', + user_id: 'user-1', + instance_id: '22222222-2222-4222-8222-222222222222', + sandbox_id: 'ki_22222222222242228222222222222222', + instance_name: 'Research Claw', + instance_destroyed_at: null, + organization_id: null, + organization_name: null, + organization_created_at: null, + organization_free_trial_end_at: null, + organization_require_seats: null, + organization_settings: null, + latest_seat_purchase_status: null, + plan: 'standard', + status: 'past_due', + email: 'user-1@example.com', + credit_renewal_at: null, + suspended_at: '2026-05-11T00:00:00.000Z', + destruction_deadline: '2026-05-18T00:00:00.000Z', + ...overrides, + }; +} + function organizationDestructionCandidateRow(overrides: Partial> = {}) { return { id: '55555555-5555-4555-8555-555555555555', @@ -1088,6 +1113,97 @@ describe('credit renewal fanout queue processing', () => { expect(staleResult.errors).toBe(0); }); + it('reconciles a duplicate suspended past-due renewal by clearing its deadline and requesting resume', async () => { + const renewalBoundary = '2026-06-01T00:00:00.000Z'; + const instanceId = '22222222-2222-4222-8222-222222222222'; + const row = creditRenewalRow({ + status: 'past_due', + past_due_since: '2026-05-01T00:00:00.000Z', + suspended_at: '2026-05-15T00:00:00.000Z', + destruction_deadline: '2026-06-02T00:00:00.000Z', + auto_resume_attempt_count: 2, + }); + const after = { + ...row, + status: 'active', + past_due_since: null, + current_period_start: renewalBoundary, + current_period_end: '2026-07-01T00:00:00.000Z', + credit_renewal_at: '2026-07-01T00:00:00.000Z', + destruction_deadline: null, + }; + const { db, deletes, txInserts, txUpdates, updates } = createMockDb( + [ + [row], + [row], + [row], + [{ id: instanceId, sandbox_id: 'ki_22222222222242228222222222222222' }], + ], + { + txInsertRowCounts: [0], + txUpdateReturningRows: [[after]], + } + ); + mockGetWorkerDb.mockReturnValue(db); + vi.spyOn(globalThis, 'fetch').mockResolvedValue( + Response.json({ + affiliateSaleEnqueued: false, + winningTouchType: 'none', + conversionId: null, + disqualificationReason: 'no_touch', + }) + ); + const kiloclawFetch = vi.fn().mockResolvedValue(Response.json({ ok: true })); + + const summary = await processCreditRenewalItem( + createEnv(kiloclawFetch), + creditRenewalItemMessage({ renewalBoundary, subscriptionId: row.id }), + 2 + ); + + expect(summary.credit_renewals_skipped_duplicate).toBe(1); + expect(summary.errors).toBe(0); + expect(kiloclawFetch).toHaveBeenCalledTimes(1); + expect(txUpdates).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + status: 'active', + past_due_since: null, + destruction_deadline: null, + }), + ]) + ); + const renewalUpdate = txUpdates.find(update => update.status === 'active'); + expect(renewalUpdate).not.toHaveProperty('suspended_at'); + expect(updates).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + auto_resume_attempt_count: 3, + auto_resume_requested_at: expect.any(String), + auto_resume_retry_after: expect.any(String), + }), + ]) + ); + expect(deletes).toHaveLength(0); + expect(txInserts).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + reason: 'credit_renewal_duplicate_idempotency_reconciled', + before_state: expect.objectContaining({ + status: 'past_due', + suspended_at: '2026-05-15T00:00:00.000Z', + destruction_deadline: '2026-06-02T00:00:00.000Z', + }), + after_state: expect.objectContaining({ + status: 'active', + suspended_at: '2026-05-15T00:00:00.000Z', + destruction_deadline: null, + }), + }), + ]) + ); + }); + it('re-reads an item when the diagnostic userId does not match the current subscription owner', async () => { const row = creditRenewalRow({ user_id: 'actual-user' }); const { db, txInserts, txUpdates, updates, selectBuilders } = createMockDb([[row]]); @@ -1176,32 +1292,23 @@ describe('credit renewal fanout queue processing', () => { ); }); - it('resolves a terminal failure when an operator retry finalizes a duplicate boundary', async () => { + it('skips duplicate side effects when guarded reconciliation loses the current boundary', async () => { const row = creditRenewalRow({ + status: 'past_due', + suspended_at: '2026-05-15T00:00:00.000Z', credit_renewal_at: '2026-06-01T00:00:00.000Z', }); - const { db, updates } = createMockDb([[row], [row], []], { + const { db, inserts, txInserts, updates } = createMockDb([[row], [row], [row]], { txInsertRowCounts: [0], txUpdateReturningRows: [[]], }); mockGetWorkerDb.mockReturnValue(db); - vi.spyOn(globalThis, 'fetch').mockImplementation(async (_request, init) => { - const body = JSON.parse(typeof init?.body === 'string' ? init.body : '{}') as { - action?: string; - }; - if (body.action === 'process_paid_conversion') { - return Response.json({ - affiliateSaleEnqueued: false, - winningTouchType: null, - conversionId: null, - disqualificationReason: 'no_touch', - }); - } - return Response.json({ ok: true }); - }); + const sideEffectFetch = vi.fn(); + vi.spyOn(globalThis, 'fetch').mockImplementation(sideEffectFetch); + const platformFetch = vi.fn(); const summary = await processCreditRenewalItem( - createEnvWithQueueMocks(vi.fn()).env, + createEnvWithQueueMocks(platformFetch).env, { kind: 'credit_renewal_item', runId: 'bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb', @@ -1214,15 +1321,18 @@ describe('credit renewal fanout queue processing', () => { 1 ); - expect(summary.credit_renewals_skipped_duplicate).toBe(1); - expect(updates).toContainEqual( + expect(summary.credit_renewals_skipped_duplicate).toBe(0); + expect(summary.credit_renewals).toBe(0); + expect(summary.errors).toBe(0); + expect(sideEffectFetch).not.toHaveBeenCalled(); + expect(platformFetch).not.toHaveBeenCalled(); + expect(updates).toHaveLength(0); + expect(inserts).toHaveLength(0); + expect(txInserts).toEqual([ expect.objectContaining({ - status: 'resolved', - resolution_actor_type: 'system', - resolution_actor_id: 'billing-lifecycle-job', - resolution_reason: 'credit_renewal_duplicate_idempotency_reconciled', - }) - ); + credit_category: 'kiloclaw-subscription:22222222-2222-4222-8222-222222222222:2026-06', + }), + ]); }); it('serializes same-user item decisions against the current locked credit balance', async () => { @@ -1415,7 +1525,7 @@ describe('interrupted auto-resume sweep', () => { vi.spyOn(console, 'error').mockImplementation(() => undefined); }); - it('requests async start and records retry metadata on acceptance', async () => { + it('retries suspended active rows after renewal cleared their destruction deadline', async () => { const instanceId = '11111111-1111-4111-8111-111111111111'; const sandboxId = 'ki_11111111111141118111111111111111'; const { db, updates } = createMockDb([ @@ -1423,7 +1533,8 @@ describe('interrupted auto-resume sweep', () => { { user_id: 'user-1', instance_id: instanceId, - suspended_at: null, + suspended_at: '2026-04-20T10:00:00.000Z', + destruction_deadline: null, auto_resume_requested_at: '2026-04-21T10:00:00.000Z', auto_resume_retry_after: '2026-04-21T12:00:00.000Z', auto_resume_attempt_count: 0, @@ -2380,6 +2491,48 @@ describe('destruction warning sweep', () => { }); }); + it('does not warn after a selected personal candidate renews and clears its deadline', async () => { + const row = { + id: 'renewed-warning-sub', + user_id: 'renewed-warning-user', + email: 'renewed-warning-user@example.com', + destruction_deadline: '2099-04-15T10:00:00.000Z', + instance_id: '12121212-1212-4212-8212-121212121212', + instance_name: 'Recovered Claw', + instance_destroyed_at: null, + organization_id: null, + plan: 'standard', + status: 'past_due', + credit_renewal_at: '2099-04-01T10:00:00.000Z', + }; + const { db, inserts } = createMockDb([[row], []]); + mockGetWorkerDb.mockReturnValue(db); + + const summary = await runSweep( + createEnv(vi.fn()), + { + runId: '14141414-1414-4414-8414-141414141414', + sweep: 'destruction_warning', + }, + 1 + ); + + expect(summary.errors).toBe(0); + expect(summary.destruction_warnings).toBe(0); + expect(summary.emails_sent).toBe(0); + expect(inserts).toHaveLength(0); + expect(globalThis.fetch).not.toHaveBeenCalled(); + expect(loggedValues).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + message: 'Skipping personal destruction warning because candidate is no longer eligible', + reason: 'candidate_no_longer_eligible', + subscriptionId: row.id, + }), + ]) + ); + }); + it('does not send destruction warning when joined instance is destroyed', async () => { const { db, inserts } = createMockDb([ [ @@ -3020,6 +3173,46 @@ describe('instance destruction sweep', () => { ); }); + it('revalidates a selected personal candidate and skips destroy after concurrent renewal', async () => { + const staleCandidate = personalDestructionCandidateRow({ + id: 'concurrent-renewal-sub', + instance_id: '13131313-1313-4313-8313-131313131313', + sandbox_id: 'ki_13131313131343138313131313131313', + status: 'past_due', + credit_renewal_at: '2026-05-01T00:00:00.000Z', + }); + const { db, deletes, inserts, txUpdates, updates } = createMockDb([[staleCandidate], []]); + mockGetWorkerDb.mockReturnValue(db); + const platformFetch = vi.fn(); + + const summary = await runSweep( + createEnv(platformFetch), + { + runId: 'bcbcbcbc-bcbc-4cbc-8cbc-bcbcbcbcbcbc', + sweep: 'instance_destruction', + }, + 1 + ); + + expect(summary.errors).toBe(0); + expect(summary.sweep3_instance_destruction).toBe(0); + expect(platformFetch).not.toHaveBeenCalled(); + expect(globalThis.fetch).not.toHaveBeenCalled(); + expect(updates).toHaveLength(0); + expect(txUpdates).toHaveLength(0); + expect(inserts).toHaveLength(0); + expect(deletes).toHaveLength(0); + expect(loggedValues).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + message: 'Skipping personal instance destruction because candidate is no longer eligible', + reason: 'candidate_no_longer_eligible', + subscriptionId: staleCandidate.id, + }), + ]) + ); + }); + it('clears destruction_deadline on a detached subscription row instead of starving the queue', async () => { // Regression: a subscription with `destruction_deadline < now()` but // `instance_id IS NULL` (its instance was already destroyed via some @@ -3193,17 +3386,15 @@ describe('instance destruction sweep', () => { it('keeps DB/email cleanup unchanged when platform destroy succeeds', async () => { const instanceId = '11111111-1111-4111-8111-111111111111'; + const candidate = personalDestructionCandidateRow({ + id: 'sub-1', + instance_id: instanceId, + sandbox_id: 'ki_11111111111141118111111111111111', + status: 'canceled', + }); const { db, updates, txUpdates, inserts, deletes, selectBuilders } = createMockDb([ - [ - { - id: 'sub-1', - user_id: 'user-1', - instance_id: instanceId, - sandbox_id: 'ki_11111111111141118111111111111111', - status: 'canceled', - email: 'user-1@example.com', - }, - ], + [candidate], + [candidate], [ { id: instanceId, @@ -3278,25 +3469,23 @@ describe('instance destruction sweep', () => { it('treats platform destroy 404 as already gone and continues with later rows', async () => { const firstInstanceId = '11111111-1111-4111-8111-111111111111'; const secondInstanceId = '22222222-2222-4222-8222-222222222222'; + const firstCandidate = personalDestructionCandidateRow({ + id: 'sub-1', + instance_id: firstInstanceId, + sandbox_id: 'ki_11111111111141118111111111111111', + status: 'canceled', + }); + const secondCandidate = personalDestructionCandidateRow({ + id: 'sub-2', + user_id: 'user-2', + instance_id: secondInstanceId, + sandbox_id: 'ki_22222222222242228222222222222222', + status: 'canceled', + email: 'user-2@example.com', + }); const { db, updates, txUpdates, inserts, deletes } = createMockDb([ - [ - { - id: 'sub-1', - user_id: 'user-1', - instance_id: firstInstanceId, - sandbox_id: 'ki_11111111111141118111111111111111', - status: 'canceled', - email: 'user-1@example.com', - }, - { - id: 'sub-2', - user_id: 'user-2', - instance_id: secondInstanceId, - sandbox_id: 'ki_22222222222242228222222222222222', - status: 'canceled', - email: 'user-2@example.com', - }, - ], + [firstCandidate, secondCandidate], + [firstCandidate], [ { id: firstInstanceId, @@ -3310,6 +3499,7 @@ describe('instance destruction sweep', () => { ], [], [{ id: 'sub-1', user_id: 'user-1', instance_id: firstInstanceId }], + [secondCandidate], [ { id: secondInstanceId, @@ -3389,17 +3579,15 @@ describe('instance destruction sweep', () => { it('logs pending platform cleanup and still preserves billing state transition', async () => { const instanceId = '11111111-1111-4111-8111-111111111111'; + const candidate = personalDestructionCandidateRow({ + id: 'sub-1', + instance_id: instanceId, + sandbox_id: 'ki_11111111111141118111111111111111', + status: 'canceled', + }); const { db, updates, txUpdates, inserts, deletes } = createMockDb([ - [ - { - id: 'sub-1', - user_id: 'user-1', - instance_id: instanceId, - sandbox_id: 'ki_11111111111141118111111111111111', - status: 'canceled', - email: 'user-1@example.com', - }, - ], + [candidate], + [candidate], [ { id: instanceId, @@ -3474,17 +3662,15 @@ describe('instance destruction sweep', () => { it('logs non-404 platform destroy failures and preserves billing state transition', async () => { const instanceId = '11111111-1111-4111-8111-111111111111'; + const candidate = personalDestructionCandidateRow({ + id: 'sub-1', + instance_id: instanceId, + sandbox_id: 'ki_11111111111141118111111111111111', + status: 'canceled', + }); const { db, updates, txUpdates, inserts, deletes } = createMockDb([ - [ - { - id: 'sub-1', - user_id: 'user-1', - instance_id: instanceId, - sandbox_id: 'ki_11111111111141118111111111111111', - status: 'canceled', - email: 'user-1@example.com', - }, - ], + [candidate], + [candidate], [ { id: instanceId, @@ -4925,6 +5111,56 @@ describe('credit renewal sweep affiliate tracking', () => { expect(typeof bonusCall?.input.nowIso).toBe('string'); }); + it('clears an unexpectedly stale destruction deadline on an active renewal', async () => { + const renewalAt = '2026-04-09T10:00:00.000Z'; + const before = creditRenewalRow({ + id: 'active-stale-deadline-sub', + instance_id: 'active-stale-deadline-instance', + instance_row_id: 'active-stale-deadline-instance', + credit_renewal_at: renewalAt, + current_period_end: renewalAt, + destruction_deadline: '2026-04-10T10:00:00.000Z', + }); + const after = { + ...before, + current_period_start: renewalAt, + current_period_end: '2026-05-09T10:00:00.000Z', + credit_renewal_at: '2026-05-09T10:00:00.000Z', + destruction_deadline: null, + }; + const { db, txInserts, txUpdates } = createMockDb([[before], [before], [before]], { + txUpdateReturningRows: [[], [after]], + }); + mockGetWorkerDb.mockReturnValue(db); + vi.spyOn(globalThis, 'fetch').mockResolvedValue(Response.json({ ok: true })); + + const summary = await processCreditRenewalItem( + createEnv(vi.fn()), + creditRenewalItemMessage({ + subscriptionId: before.id, + renewalBoundary: renewalAt, + }) + ); + + expect(summary.credit_renewals).toBe(1); + expect(summary.errors).toBe(0); + expect(txUpdates).toEqual( + expect.arrayContaining([expect.objectContaining({ destruction_deadline: null })]) + ); + expect(txInserts).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + subscription_id: before.id, + reason: 'credit_renewal', + before_state: expect.objectContaining({ + destruction_deadline: '2026-04-10T10:00:00.000Z', + }), + after_state: expect.objectContaining({ destruction_deadline: null }), + }), + ]) + ); + }); + it('sends insufficient-credit email without charging when balance and auto top-up are unavailable', async () => { const renewalAt = '2026-04-09T10:00:00.000Z'; const { db, inserts, updates, txInserts } = createMockDb( @@ -5036,10 +5272,11 @@ describe('credit renewal sweep affiliate tracking', () => { ]); }); - it('requests auto-resume when suspended past-due rows recover through credit renewal', async () => { + it('clears destruction scheduling and requests auto-resume when suspended past-due rows renew', async () => { const renewalAt = '2026-04-09T10:00:00.000Z'; const instanceId = '77777777-7777-4777-8777-777777777777'; - const { db, updates, txUpdates } = createMockDb( + const suspendedAt = '2026-04-08T10:00:00.000Z'; + const { db, deletes, updates, txUpdates } = createMockDb( [ [ { @@ -5059,7 +5296,8 @@ describe('credit renewal sweep affiliate tracking', () => { scheduled_plan: null, commit_ends_at: null, past_due_since: '2026-03-20T10:00:00.000Z', - suspended_at: '2026-04-08T10:00:00.000Z', + suspended_at: suspendedAt, + destruction_deadline: '2026-04-15T10:00:00.000Z', auto_resume_attempt_count: 2, auto_top_up_triggered_for_period: null, total_microdollars_acquired: 100_000_000, @@ -5136,12 +5374,23 @@ describe('credit renewal sweep affiliate tracking', () => { expect(summary.errors).toBe(0); expect(kiloclawFetch).toHaveBeenCalledTimes(1); expect(txUpdates).toEqual( - expect.arrayContaining([expect.objectContaining({ status: 'active', past_due_since: null })]) + expect.arrayContaining([ + expect.objectContaining({ + status: 'active', + past_due_since: null, + destruction_deadline: null, + }), + ]) ); + const renewalUpdate = txUpdates.find(update => update.status === 'active'); + expect(renewalUpdate).not.toHaveProperty('suspended_at'); const autoResumeUpdate = updates.find(update => update.auto_resume_attempt_count === 3); expect(autoResumeUpdate).toMatchObject({ auto_resume_attempt_count: 3 }); expect(typeof autoResumeUpdate?.auto_resume_requested_at).toBe('string'); expect(typeof autoResumeUpdate?.auto_resume_retry_after).toBe('string'); + expect(autoResumeUpdate).not.toHaveProperty('suspended_at'); + expect(autoResumeUpdate).not.toHaveProperty('destruction_deadline'); + expect(deletes).toHaveLength(0); }); it('normalizes Postgres renewal timestamps before paid-conversion side effects', async () => { diff --git a/services/kiloclaw-billing/src/lifecycle.ts b/services/kiloclaw-billing/src/lifecycle.ts index ecdccc70be..e0c3f85607 100644 --- a/services/kiloclaw-billing/src/lifecycle.ts +++ b/services/kiloclaw-billing/src/lifecycle.ts @@ -222,16 +222,36 @@ type OrganizationRecoveryRow = OrganizationEntitlementLifecycleFields & { instance_id: string | null; }; -type OrganizationDestructionRow = OrganizationRecoveryRow & { +type InstanceDestructionRow = { + id: string; + user_id: string; + instance_id: string | null; sandbox_id: string | null; instance_name: string | null; instance_destroyed_at: string | null; + organization_id: string | null; + organization_name: string | null; plan: KiloClawPlan; status: KiloClawSubscriptionStatus; email: string; credit_renewal_at: string | null; }; +type OrganizationDestructionRow = OrganizationEntitlementLifecycleFields & InstanceDestructionRow; + +type PersonalDestructionWarningRow = { + id: string; + user_id: string; + email: string; + destruction_deadline: string | null; + instance_id: string; + instance_name: string | null; + instance_destroyed_at: string | null; + organization_id: string | null; + plan: KiloClawPlan; + credit_renewal_at: string | null; +}; + type KiloPassProjectionSubscriptionRow = KiloPassBonusProjectionSubscription & KiloPassSubscriptionProjectionCandidate & { id: string; @@ -1590,6 +1610,35 @@ async function autoResumeIfSuspended( return true; } +async function handleCreditRenewalRecoveryAfterAdvance( + env: BillingWorkerEnv, + database: WorkerDb, + context: SweepExecutionContext, + row: CreditRenewalRow, + wasPastDue: boolean +): Promise { + if (!wasPastDue) return; + + if (!row.suspended_at) { + await database.delete(kiloclaw_email_log).where( + emailLogRowCondition({ + userId: row.user_id, + instanceId: row.instance_id, + emailType: 'claw_credit_renewal_failed', + }) + ); + return; + } + + await autoResumeIfSuspended(env, database, context, { + id: row.id, + user_id: row.user_id, + instance_id: row.instance_id, + organization_id: row.organization_id, + auto_resume_attempt_count: row.auto_resume_attempt_count, + }); +} + type CreditRenewalTransactionOutcome = | { kind: 'skipped' } | { kind: 'canceled'; row: CreditRenewalRow; renewalAt: string } @@ -1601,6 +1650,7 @@ type CreditRenewalTransactionOutcome = effectivePlan: 'commit' | 'standard'; priceVersion: string; costMicrodollars: number; + wasPastDue: boolean; row: CreditRenewalRow; newPeriodEnd: string; } @@ -1658,6 +1708,7 @@ function buildCreditRenewalAdvanceUpdateSet(params: { current_period_start: params.newPeriodStart, current_period_end: params.newPeriodEnd, credit_renewal_at: params.newPeriodEnd, + destruction_deadline: null, auto_top_up_triggered_for_period: null, }; @@ -1862,27 +1913,29 @@ async function processCreditRenewalRow( ) .returning(); - if (updatedSubscription) { - await insertKiloClawSubscriptionChangeLog(tx, { - subscriptionId: current.id, - actor: LIFECYCLE_ACTOR, - action: changeAction, - reason: 'credit_renewal_duplicate_idempotency_reconciled', - before: beforeSubscription, - after: updatedSubscription, - }); - - await supersedeTerminalRenewalFailuresForBoundary(tx, { - subscriptionId: current.id, - currentBoundary: newPeriodEnd, - actor: { - type: LIFECYCLE_ACTOR.actorType, - id: LIFECYCLE_ACTOR.actorId, - }, - supersededAt: new Date().toISOString(), - }); + if (!updatedSubscription) { + return { kind: 'skipped' } satisfies CreditRenewalTransactionOutcome; } + await insertKiloClawSubscriptionChangeLog(tx, { + subscriptionId: current.id, + actor: LIFECYCLE_ACTOR, + action: changeAction, + reason: 'credit_renewal_duplicate_idempotency_reconciled', + before: beforeSubscription, + after: updatedSubscription, + }); + + await supersedeTerminalRenewalFailuresForBoundary(tx, { + subscriptionId: current.id, + currentBoundary: newPeriodEnd, + actor: { + type: LIFECYCLE_ACTOR.actorType, + id: LIFECYCLE_ACTOR.actorId, + }, + supersededAt: new Date().toISOString(), + }); + return { kind: 'duplicate', userId, @@ -1891,6 +1944,7 @@ async function processCreditRenewalRow( effectivePlan, priceVersion: current.kiloclaw_price_version, costMicrodollars, + wasPastDue, row: current, newPeriodEnd, } satisfies CreditRenewalTransactionOutcome; @@ -1970,6 +2024,14 @@ async function processCreditRenewalRow( } if (outcome.kind === 'duplicate') { + await handleCreditRenewalRecoveryAfterAdvance( + env, + database, + context, + outcome.row, + outcome.wasPastDue + ); + await processPaidConversionBestEffort(env, context, { userId: outcome.userId, dedupeKey: `affiliate:impact:sale:${outcome.deductionCategory}`, @@ -2015,25 +2077,13 @@ async function processCreditRenewalRow( }); } - if (outcome.wasPastDue && !outcome.row.suspended_at) { - await database.delete(kiloclaw_email_log).where( - emailLogRowCondition({ - userId: outcome.userId, - instanceId: outcome.row.instance_id, - emailType: 'claw_credit_renewal_failed', - }) - ); - } - - if (outcome.wasPastDue && outcome.row.suspended_at) { - await autoResumeIfSuspended(env, database, context, { - id: outcome.row.id, - user_id: outcome.userId, - instance_id: outcome.row.instance_id, - organization_id: outcome.row.organization_id, - auto_resume_attempt_count: outcome.row.auto_resume_attempt_count, - }); - } + await handleCreditRenewalRecoveryAfterAdvance( + env, + database, + context, + outcome.row, + outcome.wasPastDue + ); await processPaidConversionBestEffort(env, context, { userId: outcome.userId, @@ -2984,6 +3034,102 @@ async function loadCurrentOrganizationTrialExpiryRow( return row ?? null; } +async function loadCurrentPersonalDestructionRow( + database: Pick, + params: { + subscriptionId: string; + userId: string; + instanceId: string; + sandboxId: string; + now: string; + } +): Promise { + const [row] = await database + .select({ + id: kiloclaw_subscriptions.id, + user_id: kiloclaw_subscriptions.user_id, + instance_id: kiloclaw_subscriptions.instance_id, + sandbox_id: kiloclaw_instances.sandbox_id, + instance_name: kiloclaw_instances.name, + instance_destroyed_at: kiloclaw_instances.destroyed_at, + organization_id: kiloclaw_instances.organization_id, + organization_name: sql`null`.as('organization_name'), + plan: kiloclaw_subscriptions.plan, + status: kiloclaw_subscriptions.status, + email: kilocode_users.google_user_email, + credit_renewal_at: kiloclaw_subscriptions.credit_renewal_at, + }) + .from(kiloclaw_subscriptions) + .innerJoin(kilocode_users, eq(kiloclaw_subscriptions.user_id, kilocode_users.id)) + .innerJoin(kiloclaw_instances, eq(kiloclaw_subscriptions.instance_id, kiloclaw_instances.id)) + .where( + and( + eq(kiloclaw_subscriptions.id, params.subscriptionId), + eq(kiloclaw_subscriptions.user_id, params.userId), + eq(kiloclaw_subscriptions.instance_id, params.instanceId), + eq(kiloclaw_instances.id, params.instanceId), + eq(kiloclaw_instances.user_id, params.userId), + eq(kiloclaw_instances.sandbox_id, params.sandboxId), + lt(kiloclaw_subscriptions.destruction_deadline, params.now), + currentSubscriptionRowFilter(), + isNotNull(kiloclaw_subscriptions.suspended_at), + inArray(kiloclaw_subscriptions.status, ['canceled', 'past_due', 'unpaid']), + isNull(kiloclaw_instances.destroyed_at), + isNull(kiloclaw_instances.organization_id) + ) + ) + .limit(1); + + return row ?? null; +} + +async function loadCurrentPersonalDestructionWarningRow( + database: Pick, + params: { + subscriptionId: string; + userId: string; + instanceId: string; + advisoryNow: string; + warningCutoff: string; + } +): Promise { + const [row] = await database + .select({ + id: kiloclaw_subscriptions.id, + user_id: kiloclaw_subscriptions.user_id, + email: kilocode_users.google_user_email, + destruction_deadline: kiloclaw_subscriptions.destruction_deadline, + instance_id: kiloclaw_instances.id, + instance_name: kiloclaw_instances.name, + instance_destroyed_at: kiloclaw_instances.destroyed_at, + organization_id: kiloclaw_instances.organization_id, + plan: kiloclaw_subscriptions.plan, + credit_renewal_at: kiloclaw_subscriptions.credit_renewal_at, + }) + .from(kiloclaw_subscriptions) + .innerJoin(kilocode_users, eq(kiloclaw_subscriptions.user_id, kilocode_users.id)) + .innerJoin(kiloclaw_instances, eq(kiloclaw_subscriptions.instance_id, kiloclaw_instances.id)) + .where( + and( + eq(kiloclaw_subscriptions.id, params.subscriptionId), + eq(kiloclaw_subscriptions.user_id, params.userId), + eq(kiloclaw_subscriptions.instance_id, params.instanceId), + eq(kiloclaw_instances.id, params.instanceId), + eq(kiloclaw_instances.user_id, params.userId), + gte(kiloclaw_subscriptions.destruction_deadline, params.advisoryNow), + lte(kiloclaw_subscriptions.destruction_deadline, params.warningCutoff), + currentSubscriptionRowFilter(), + isNotNull(kiloclaw_subscriptions.suspended_at), + inArray(kiloclaw_subscriptions.status, ['canceled', 'past_due', 'unpaid']), + isNull(kiloclaw_instances.destroyed_at), + isNull(kiloclaw_instances.organization_id) + ) + ) + .limit(1); + + return row ?? null; +} + async function loadCurrentOrganizationDestructionRow( database: Pick, subscriptionId: string, @@ -3775,7 +3921,7 @@ async function runInstanceDestructionSweep( continue; } - let destructionRow = row; + let destructionRow: InstanceDestructionRow = row; if (row.organization_id) { const currentRow = await loadCurrentOrganizationDestructionRow(database, row.id, now); if (!currentRow) { @@ -3827,6 +3973,25 @@ async function runInstanceDestructionSweep( ); continue; } + } else { + const currentRow = await loadCurrentPersonalDestructionRow(database, { + subscriptionId: row.id, + userId: row.user_id, + instanceId: row.instance_id, + sandboxId: row.sandbox_id, + now, + }); + if (!currentRow) { + logSkippedSubscriptionRow( + 'Skipping personal instance destruction because candidate is no longer eligible', + row, + { + reason: 'candidate_no_longer_eligible', + } + ); + continue; + } + destructionRow = currentRow; } const destructionInstanceId = destructionRow.instance_id; @@ -4288,31 +4453,50 @@ async function runDestructionWarningSweep( ); continue; } - if (await hasUnresolvedTerminalRenewalFailureForBoundary(database, row)) { + + const currentRow = await loadCurrentPersonalDestructionWarningRow(database, { + subscriptionId: row.id, + userId: row.user_id, + instanceId: row.instance_id, + advisoryNow, + warningCutoff: twoDaysFromNow, + }); + if (!currentRow?.destruction_deadline) { + logSkippedSubscriptionRow( + 'Skipping personal destruction warning because candidate is no longer eligible', + row, + { + reason: 'candidate_no_longer_eligible', + } + ); + continue; + } + + if (await hasUnresolvedTerminalRenewalFailureForBoundary(database, currentRow)) { continue; } - const instanceIdShort = shortInstanceId(row.instance_id); + const instanceIdShort = shortInstanceId(currentRow.instance_id); const sent = await trySendEmail( database, env, context, - row.user_id, - row.email, + currentRow.user_id, + currentRow.email, 'claw_destruction_warning', 'clawDestructionWarning', { - destruction_date: formatDateForEmail(new Date(row.destruction_deadline)), + destruction_date: formatDateForEmail(new Date(currentRow.destruction_deadline)), claw_url: clawUrl, instance_label: formatInstanceLabel({ - instanceName: row.instance_name, - instanceId: row.instance_id, - plan: row.plan, + instanceName: currentRow.instance_name, + instanceId: currentRow.instance_id, + plan: currentRow.plan, }), instance_id_short: instanceIdShort, }, summary, undefined, - { instanceId: row.instance_id } + { instanceId: currentRow.instance_id } ); if (sent) summary.destruction_warnings++; } catch (error) {