Skip to content

Advance ack level when queue is drained#9731

Merged
dnr merged 4 commits intotemporalio:mainfrom
dnr:reset
Apr 7, 2026
Merged

Advance ack level when queue is drained#9731
dnr merged 4 commits intotemporalio:mainfrom
dnr:reset

Conversation

@dnr
Copy link
Copy Markdown
Contributor

@dnr dnr commented Mar 28, 2026

What changed?

In priBacklogManager, use the isDrained logic (outstandingTasks) to move the ack level up when a queue is empty.

Why?

If the task id range ends in a gap, we won't be able to move the ack level higher based on completed tasks alone, but we can move it up when the queue is totally empty. This also lets us trigger the backlog count divergence reset.

How did you test it?

  • built
  • run locally and tested manually
  • covered by existing tests
  • added new unit test(s)
  • added new functional test(s)

@dnr dnr requested a review from a team as a code owner March 28, 2026 01:51
Comment thread service/matching/db.go Outdated

if newAckLevel == db.getMaxReadLevelLocked(subqueue) {
// Reset approximateBacklogCount to fix the count divergence issue
if newAckLevel == db.getMaxReadLevelLocked(subqueue) || isDrained {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of adding this isDrained check, can we not advance ackLevel to readLevel when outstandingTasks is empty? This makes the ackLevel accurate and existing reset check should work.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory, yes. I felt like this was a more conservative change and that would be more risky: if something was wrong with the logic, just adjusting the backlog count is safer than skipping over tasks. But it's pretty clear: if we've read to the end and have no tasks, then we can move the ack level. I think I see where to put the check, let me try it that way...

@dnr dnr changed the title Improve reset backlog count divergence Advance ack level when queue is drained Apr 7, 2026
@dnr dnr merged commit a78e744 into temporalio:main Apr 7, 2026
47 checks passed
@dnr dnr deleted the reset branch April 7, 2026 06:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants