Skip to content

Conversation

@Brend-Smits
Copy link
Contributor

@Brend-Smits Brend-Smits commented Jan 7, 2026

This pull request enhances the robustness and reliability of the GitHub Actions runner scaling logic by improving error handling and retry mechanisms for GitHub API calls. It introduces the @octokit/plugin-retry plugin to automatically retry failed API requests, adds detailed logging for retry attempts, and ensures that failures in creating JIT configs for individual runner instances do not halt the entire scaling process. Additionally, new tests are added to verify handling of various API failure scenarios.

GitHub API client improvements:

  • Added @octokit/plugin-retry to dependencies (package.json) and integrated it into the Octokit client initialization to enable automatic retries for failed GitHub API requests. [1] [2] [3]
  • Configured the retry plugin to log detailed warnings on each retry attempt, including the HTTP method, URL, error message, and status code.

Error handling and resilience in JIT config creation:

  • Updated createJitConfig in scale-up.ts to catch and log errors for individual runner instances when creating JIT configs, allowing the process to continue for remaining instances and logging a summary of failed attempts at the end. [1] [2]
  • Instances that failed to generate a configuration, will now be terminated to avoid generating waste.

Testing improvements:

  • Added comprehensive tests to scale-up.test.ts to verify correct behavior when GitHub API calls fail for some instances, including retryable errors (e.g., 5xx), non-retryable errors (e.g., 4xx), and partial failures, ensuring only successful JIT configs are stored.

This ensures that even if there's a failed jit config creation for one of the instances, it proceeds with the other ones and does not just skip the entire batch. It will report the failed instances at the end.
@Brend-Smits Brend-Smits requested a review from a team as a code owner January 7, 2026 12:22
@github-actions
Copy link
Contributor

github-actions bot commented Jan 7, 2026

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

OpenSSF Scorecard

PackageVersionScoreDetails
npm/@octokit/plugin-retry 8.0.3 🟢 7.6
Details
CheckScoreReason
Code-Review🟢 10all changesets reviewed
Security-Policy🟢 9security policy file detected
Binary-Artifacts🟢 10no binaries found in the repo
Maintained🟢 911 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 9
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
Token-Permissions⚠️ 0detected GitHub workflow tokens with excessive permissions
Pinned-Dependencies🟢 5dependency not pinned by hash detected -- score normalized to 5
License🟢 10license file detected
Fuzzing⚠️ 0project is not fuzzed
Signed-Releases⚠️ -1no releases found
Branch-Protection⚠️ -1internal error: error during branchesHandler.setup: internal error: some github tokens can't read classic branch protection rules: https://github.com/ossf/scorecard-action/blob/main/docs/authentication/fine-grained-auth-token.md
Vulnerabilities🟢 100 existing vulnerabilities detected
Packaging🟢 10packaging workflow detected
SAST🟢 10SAST tool is run on all commits
npm/@octokit/plugin-retry 8.0.3 🟢 7.6
Details
CheckScoreReason
Code-Review🟢 10all changesets reviewed
Security-Policy🟢 9security policy file detected
Binary-Artifacts🟢 10no binaries found in the repo
Maintained🟢 911 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 9
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
Token-Permissions⚠️ 0detected GitHub workflow tokens with excessive permissions
Pinned-Dependencies🟢 5dependency not pinned by hash detected -- score normalized to 5
License🟢 10license file detected
Fuzzing⚠️ 0project is not fuzzed
Signed-Releases⚠️ -1no releases found
Branch-Protection⚠️ -1internal error: error during branchesHandler.setup: internal error: some github tokens can't read classic branch protection rules: https://github.com/ossf/scorecard-action/blob/main/docs/authentication/fine-grained-auth-token.md
Vulnerabilities🟢 100 existing vulnerabilities detected
Packaging🟢 10packaging workflow detected
SAST🟢 10SAST tool is run on all commits

Scanned Files

  • lambdas/functions/control-plane/package.json
  • lambdas/yarn.lock

@Brend-Smits Brend-Smits force-pushed the fix/createJitConfig-error-handling branch from cd7ca8f to 9f37a04 Compare January 7, 2026 13:07
Instances that failed to start up because of incorrect configuration never got terminated. This is now updated and failed instances get terminated right away. Previously we relied on a scale-down to do this.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants