Skip to content

feat(webhook): use priority field as tiebreaker in matcher dispatch#5133

Draft
vegardx wants to merge 1 commit into
github-aws-runners:mainfrom
vegardx:feat/priority-dispatch-matcher-config
Draft

feat(webhook): use priority field as tiebreaker in matcher dispatch#5133
vegardx wants to merge 1 commit into
github-aws-runners:mainfrom
vegardx:feat/priority-dispatch-matcher-config

Conversation

@vegardx
Copy link
Copy Markdown

@vegardx vegardx commented May 26, 2026

Problem

When multiple runner pools match the same job labels via exactMatch: true (subset matching), dispatch order is non-deterministic. A workflow requesting [self-hosted, linux, x64] can land on any pool whose labels contain that subset — including an expensive 16 vCPU on-demand pool when a cheap 2 vCPU spot pool would suffice.

See #5128 for the full discussion and concrete examples.

Fix

The Terraform variable runner_matcher_config already supports an optional priority field (default 999) and pre-sorts the config before writing to SSM. However the Lambda re-sorts only by exactMatch, discarding the priority ordering. This PR adds priority as a secondary sort key:

matcherConfig.sort((a, b) => {
  if (a.matcherConfig.exactMatch !== b.matcherConfig.exactMatch) {
    return a.matcherConfig.exactMatch ? -1 : 1;
  }
  return (a.matcherConfig.priority ?? 999) - (b.matcherConfig.priority ?? 999);
});

The assumption is that when a workflow omits specific labels (e.g. uses just [self-hosted, linux, x64] instead of [self-hosted, linux, x64, large, ondemand]), the omission is intentional — the requester does not care which specific pool handles the job. The priority field lets the operator decide which pool should be preferred for these ambiguous requests.

Changes

  • lambdas/functions/webhook/src/sqs/index.ts — add optional priority field to MatcherConfig
  • lambdas/functions/webhook/src/runners/dispatch.ts — sort by priority as tiebreaker
  • lambdas/functions/webhook/src/runners/dispatch.test.ts — test: multiple pools match common subset, lowest priority wins

Backwards compatible

  • priority is optional, defaults to 999
  • Existing configs without priority behave identically (all sort equal, first match wins as before)
  • The Terraform variable already defines priority = optional(number, 999) — no Terraform changes needed

Refs: #5128

When multiple runner pools match the same job labels (subset match via
exactMatch: true), sort by ascending priority so the lowest-priority
pool (cheapest) is selected first.

The Terraform variable already supports priority (optional, default 999)
and pre-sorts the config before writing to SSM. However the Lambda
re-sorted only by exactMatch, losing the priority ordering. This commit
adds priority as a secondary sort key in the Lambda.

Refs: github-aws-runners#5128
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant