Stop futile auth retries in Copilot/Claude/Codex harnesses after first-attempt authentication failure by Copilot · Pull Request #33093 · github/gh-aw

Copilot · 2026-05-18T16:41:34Z

Retry loops in the harnesses were optimized for transient partial-execution failures, but they also retried after first-attempt auth failures. In workflows where entrypoint cleanup removes token-based auth between attempts, this produced guaranteed follow-on failures and noisy diagnostics.

Retry policy hardening (first-attempt auth failures)
- Added explicit detection for Authentication failed (Request ID: ...) in:
  - actions/setup/js/copilot_harness.cjs
  - actions/setup/js/claude_harness.cjs
  - actions/setup/js/codex_harness.cjs
- Short-circuits retries when this error occurs on attempt 1 (attempt === 0), treating it as non-retryable.
- Preserves existing retry behavior for other transient and partial-execution paths.
Operational diagnostics
- Extended per-attempt structured log fields with isAuthenticationFailedError to make retry decisions obvious in run logs.
Focused test updates
- Added/updated harness tests to cover:
  - auth-failure pattern detection
  - no-retry behavior on first-attempt auth failure
- Updated tests to reuse exported helper detection functions where applicable to avoid duplicated regex logic.

if (attempt === 0 && isAuthenticationFailedError(result.output)) {
  log(`attempt ${attempt + 1}: authentication failed — not retrying (first-attempt auth failure is non-retryable)`);
  break;
}

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot

Pull request overview

Hardens retry behavior in the Copilot/Claude/Codex harnesses by treating first-attempt authentication failures as non-retryable, reducing noisy/futile retry loops and improving per-attempt diagnostics.

Changes:

Added isAuthenticationFailedError(...) detection + first-attempt no-retry short-circuit in copilot_harness.cjs, claude_harness.cjs, and codex_harness.cjs, and surfaced isAuthenticationFailedError in structured per-attempt logs.
Updated/added unit tests to cover the new auth-failure detection and “no retry on attempt 1” policy.
Updated .github/workflows/pr-description-caveman.lock.yml (large generated workflow lockfile changes).

Show a summary per file

File	Description
actions/setup/js/copilot_harness.cjs	Adds auth-failure detection helper, exports it, logs the new flag, and breaks retries on first-attempt auth failure.
actions/setup/js/copilot_harness.test.cjs	Adds tests for auth-failure detection and first-attempt no-retry behavior.
actions/setup/js/claude_harness.cjs	Adds auth-failure detection helper/export, logs the new flag, and breaks retries on first-attempt auth failure.
actions/setup/js/claude_harness.test.cjs	Adds tests for the new auth-failure detection and retry policy behavior.
actions/setup/js/codex_harness.cjs	Adds auth-failure detection helper/export, logs the new flag, and breaks retries on first-attempt auth failure.
actions/setup/js/codex_harness.test.cjs	Adds tests for the new auth-failure detection and first-attempt no-retry behavior.
.github/workflows/pr-description-caveman.lock.yml	Regenerates/updates the workflow lockfile (versions, steps, and runtime details).

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 7/7 changed files
Comments generated: 7

+// After a first-attempt auth failure, retrying is futile because the entrypoint unsets
+// COPILOT_GITHUB_TOKEN between attempts.
+const AUTHENTICATION_FAILED_PATTERN = /Authentication failed(?:\s*\(Request ID:[^)]+\))?/i;


 //   - embedded stream-json result fields (e.g. "api_error_status":429)
 //   - human-readable message text ("rate limit")
 const RATE_LIMIT_ERROR_PATTERN = /rate_limit_error|429 Too Many Requests|"api_error_status"\s*:\s*429|request rejected \(429\)|rate limit/i;
+const AUTHENTICATION_FAILED_PATTERN = /Authentication failed(?:\s*\(Request ID:[^)]+\))?/i;


 // Matches "rate_limit_exceeded" from the OpenAI error type field and the "429" status code
 // that Codex emits when the API rate limit is hit.
 const RATE_LIMIT_ERROR_PATTERN = /rate_limit_exceeded|429 Too Many Requests|RateLimitError/i;
+const AUTHENTICATION_FAILED_PATTERN = /Authentication failed(?:\s*\(Request ID:[^)]+\))?/i;


+    it("matches authentication failed with request id", () => {
+      expect(isAuthenticationFailedError("Authentication failed (Request ID: C818:3ED713:19D401B:1C446B7:69D653CA)")).toBe(true);
+    });
+


+      - name: Checkout actions folder
+        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        with:
+          repository: github/gh-aw
+          sparse-checkout: |
+            actions
+          persist-credentials: false
      - name: Setup Scripts
        id: setup
-        uses: github/gh-aw-actions/setup@v0.72.1
+        uses: ./actions/setup
        with:


+  describe("isAuthenticationFailedError", () => {
+    it("returns true for authentication failed with request id", () => {
+      expect(isAuthenticationFailedError("Authentication failed (Request ID: C818:3ED713:19D401B:1C446B7:69D653CA)")).toBe(true);
+    });
+
+    it("returns false for non-authentication-failed output", () => {
+      expect(isAuthenticationFailedError("No authentication information found")).toBe(false);
+      expect(isAuthenticationFailedError("rate_limit_exceeded")).toBe(false);
+    });


+
+    it("returns false for unrelated output", () => {
+      expect(isAuthenticationFailedError("No authentication information found")).toBe(false);
+      expect(isAuthenticationFailedError("rate_limit_error")).toBe(false);


Initial plan

01ba172

Copilot AI assigned Copilot and pelikhan May 18, 2026

Copilot started work on behalf of pelikhan May 18, 2026 16:41 View session

Copilot AI linked an issue May 18, 2026 that may be closed by this pull request

Copilot CLI retry loop in copilot_driver.cjs is futile after entrypoint unsets COPILOT_GITHUB_TOKEN between attempts #33069

Closed

Copilot AI and others added 3 commits May 18, 2026 16:47

Plan auth-retry fix

7c6492f

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Stop harness retries after first auth failure

ca029c3

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Deduplicate auth-failure test patterns

d528035

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Fix Copilot CLI retry loop after unsetting GitHub token~~ Stop futile auth retries in Copilot/Claude/Codex harnesses after first-attempt authentication failure May 18, 2026

Copilot finished work on behalf of pelikhan May 18, 2026 17:00

Copilot AI requested a review from pelikhan May 18, 2026 17:00

pelikhan approved these changes May 18, 2026

View reviewed changes

pelikhan marked this pull request as ready for review May 18, 2026 17:01

Copilot AI review requested due to automatic review settings May 18, 2026 17:01

pelikhan merged commit e2089e4 into main May 18, 2026

pelikhan deleted the copilot/fix-copilot-cli-retry-loop branch May 18, 2026 17:02

Copilot started reviewing on behalf of pelikhan May 18, 2026 17:03 View session

Copilot AI reviewed May 18, 2026

View reviewed changes

This was referenced May 18, 2026

Smoke Test: Codex - 26047161375 #33101

Closed

Prevent safe-output PR probing from creating stray downstream pull requests #33044

Merged

Smoke Test: Claude - 26047160386 #33103

Closed

Smoke Test: Copilot - 26047160572 #33105

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Stop futile auth retries in Copilot/Claude/Codex harnesses after first-attempt authentication failure#33093

Stop futile auth retries in Copilot/Claude/Codex harnesses after first-attempt authentication failure#33093
pelikhan merged 4 commits into
mainfrom
copilot/fix-copilot-cli-retry-loop

Copilot AI commented May 18, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Copilot AI commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Copilot's findings

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented May 18, 2026 •

edited

Loading