Enhance crontab job execution with re-entry protection#1317
Enhance crontab job execution with re-entry protection#1317adenchen123 wants to merge 3 commits intoSciSharp:masterfrom
Conversation
Review Summary by QodoAdd distributed re-entry protection for crontab job execution
WalkthroughsDescription• Add re-entry protection mechanism to prevent concurrent crontab job execution • Introduce distributed locking using IDistributedLocker for task synchronization • Add ReentryProtection boolean flag to CrontabItem model (enabled by default) • Refactor execution logic with graceful fallback when lock acquisition fails • Improve logging with detailed lock acquisition and failure messages Diagramflowchart LR
A["CrontabItem<br/>+ ReentryProtection flag"] --> B["ExecuteTimeArrivedItemWithReentryProtection"]
B --> C{"ReentryProtection<br/>enabled?"}
C -->|Yes| D["Acquire Distributed Lock"]
C -->|No| E["Execute directly"]
D --> F{"Lock<br/>acquired?"}
F -->|Yes| G["Execute Task"]
F -->|No| H["Skip execution<br/>log warning"]
D --> I{"Exception<br/>occurred?"}
I -->|Before lock| J["Execute without lock"]
I -->|After lock| K["Log warning"]
G --> L["Release Lock"]
E --> M["Complete"]
J --> M
K --> M
File Changes1. src/Infrastructure/BotSharp.Abstraction/Crontab/ICrontabService.cs
|
Code Review by Qodo
1. Lock key too coarse
|
| var lockKey = $"crontab:execution:{item.Title}"; | ||
| using var scope = _services.CreateScope(); | ||
| var locker = scope.ServiceProvider.GetRequiredService<IDistributedLocker>(); | ||
| var acquired = false; | ||
| var lockAcquired = false; | ||
|
|
||
| try | ||
| { | ||
| acquired = await locker.LockAsync(lockKey, async () => | ||
| { | ||
| lockAcquired = true; | ||
| _logger.LogInformation("Crontab: {0}, Distributed lock acquired, beginning execution...", item.Title); | ||
| await ExecuteTimeArrivedItem(item); | ||
| }, timeout: 600); | ||
|
|
||
| if (!acquired) | ||
| { | ||
| _logger.LogWarning("Crontab: {0}, Failed to acquire distributed lock, task is still executing, skipping this occurrence to prevent re-entry.", item.Title); | ||
| } |
There was a problem hiding this comment.
1. Lock key too coarse 🐞 Bug ≡ Correctness
ExecuteTimeArrivedItemWithReentryProtection builds the distributed lock key from only CrontabItem.Title, so different crontab items (e.g., different ConversationId/AgentId) sharing the same Title will block each other and cause skipped executions.
Agent Prompt
### Issue description
`ExecuteTimeArrivedItemWithReentryProtection` uses `crontab:execution:{item.Title}` as the distributed lock key. Since crontab items are persisted/upserted by `ConversationId` (and items also vary by `AgentId`/`UserId`), different jobs can share a title and incorrectly block each other, leading to skipped executions.
### Issue Context
- Crontab items in Mongo are upserted by `ConversationId`, not by `Title`, so `Title` is not a safe global unique identifier.
- The lock key should incorporate the job's real identity (at least `ConversationId`, and likely `AgentId` too).
### Fix Focus Areas
- src/Infrastructure/BotSharp.Core.Crontab/Services/CrontabService.cs[132-158]
### Implementation notes
- Build the lock key from stable identifiers, e.g.:
- `crontab:execution:{item.TriggerType}:{item.AgentId}:{item.ConversationId}:{item.Title}`
- If some sources don’t have `ConversationId`, fall back to a safe placeholder like `global`.
- Keep the log message consistent with the new key (include key parts for debugging).
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
…te CrontabItemDocument to include TriggerType and ReentryProtection properties.
No description provided.