Add automated duplicate for new issue detection and auto-close workflows

.claude/commands/dedupe.md

@@ -0,0 +1,33 @@
+---
+allowed-tools: Bash(gh:*), Bash(./scripts/comment-on-duplicates.sh:*)
+description: Find duplicate GitHub issues
+---
+
+Find up to 3 likely duplicate issues for a given GitHub issue.
+
+Follow these steps precisely:
+
+1. Use `gh issue view <number>` to read the issue. If the issue is closed, or is broad product feedback without a specific bug/feature request, or already has a duplicate detection comment (containing `<!-- duplicate-detection -->`), stop and report why you are not proceeding.
+
+2. Summarize the issue's core problem in 2-3 sentences. Identify the key terms, error messages, and affected components.
+
+3. Search for potential duplicates using **at least 3 different search strategies**. Run these searches in parallel. **Only consider issues with a lower issue number** (older issues) as potential originals — skip any result with a number >= the current issue. Also skip issues already labeled `duplicate`.
+   - `gh search issues "<exact error message or key phrase>" --repo $GITHUB_REPOSITORY --state open -- -label:duplicate --limit 15 --json number,title | jq '[.[] | select(.number < <current-issue-number>)]'`
+   - `gh search issues "<component or feature keywords>" --repo $GITHUB_REPOSITORY --state open -- -label:duplicate --limit 15 --json number,title | jq '[.[] | select(.number < <current-issue-number>)]'`
+   - `gh search issues "<alternate description of the problem>" --repo $GITHUB_REPOSITORY --state open -- -label:duplicate --limit 15 --json number,title | jq '[.[] | select(.number < <current-issue-number>)]'`
+   - `gh search issues "<key terms>" --repo $GITHUB_REPOSITORY --state all -- -label:duplicate --limit 10 --json number,title | jq '[.[] | select(.number < <current-issue-number>)]'` (include closed issues for reference)
+
+4. For each candidate issue that looks like a potential match, read it with `gh issue view <number>` to verify it is truly about the same problem. Filter out false positives — issues that merely share keywords but describe different problems.
+
+5. If you find 1-3 genuine duplicates, post the result using the comment script:
+   ```
+   ./scripts/comment-on-duplicates.sh --base-issue <issue-number> --potential-duplicates <dup1> [dup2] [dup3]
+   ```
+
+6. If no genuine duplicates are found, report that no duplicates were detected and take no further action.
+
+Important notes:
+- Only flag issues as duplicates when you are confident they describe the **same underlying problem**
+- Prefer open issues as duplicates, but closed issues can be referenced too
+- Do not flag the issue as a duplicate of itself
+- The base issue number is the last part of the issue reference (e.g., for `owner/repo/issues/42`, the number is `42`)

.github/workflows/auto-close-duplicates.yml

@@ -0,0 +1,132 @@
+name: Auto-close duplicate issues
+
+on:
+  schedule:
+    - cron: "0 9 * * *"
+  workflow_dispatch:
+
+permissions:
+  issues: write
+
+jobs:
+  auto-close-duplicates:
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+    steps:
+      - name: Close stale duplicate issues
+        uses: actions/github-script@v7
+        env:
+          GRACE_DAYS: ${{ vars.DUPLICATE_GRACE_DAYS || '7' }}
+        with:
+          script: |
+            const { owner, repo } = context.repo;
+            const graceDays = parseInt(process.env.GRACE_DAYS, 10) || 7;
+            const GRACE_PERIOD_MS = graceDays * 24 * 60 * 60 * 1000;
+            const now = Date.now();
+
+            // Find all open issues with the duplicate label
+            const issues = await github.paginate(github.rest.issues.listForRepo, {
+              owner,
+              repo,
+              state: 'open',
+              labels: 'duplicate',
+              per_page: 100,
+            });
+
+            console.log(`Found ${issues.length} open issues with duplicate label`);
+
+            let closedCount = 0;
+
+            for (const issue of issues) {
+              console.log(`Processing issue #${issue.number}: ${issue.title}`);
+
+              // Get comments to find the duplicate detection comment
+              const comments = await github.rest.issues.listComments({
+                owner,
+                repo,
+                issue_number: issue.number,
+                per_page: 100,
+              });
+
+              // Find the duplicate detection comment (posted by our script)
+              const dupeComments = comments.data.filter(c =>
+                c.body.includes('<!-- duplicate-detection -->')
+              );
+
+              if (dupeComments.length === 0) {
+                console.log(`  No duplicate detection comment found, skipping`);
+                continue;
+              }
+
+              const lastDupeComment = dupeComments[dupeComments.length - 1];
+              const dupeCommentAge = now - new Date(lastDupeComment.created_at).getTime();
+
+              if (dupeCommentAge < GRACE_PERIOD_MS) {
+                const daysLeft = ((GRACE_PERIOD_MS - dupeCommentAge) / (24 * 60 * 60 * 1000)).toFixed(1);
+                console.log(`  Duplicate comment is too recent (${daysLeft} days remaining), skipping`);
+                continue;
+              }
+
+              // Check for human comments after the duplicate detection comment
+              const humanCommentsAfter = comments.data.filter(c =>
+                new Date(c.created_at) > new Date(lastDupeComment.created_at) &&
+                c.user.type !== 'Bot' &&
+                !c.body.includes('<!-- duplicate-detection -->') &&
+                !c.body.includes('automatically closed as a duplicate')
+              );
+
+              if (humanCommentsAfter.length > 0) {
+                console.log(`  Has ${humanCommentsAfter.length} human comment(s) after detection, skipping`);
+                continue;
+              }
+
+              // Check for thumbs-down reaction from the issue author
+              const reactions = await github.rest.reactions.listForIssueComment({
+                owner,
+                repo,
+                comment_id: lastDupeComment.id,
+                per_page: 100,
+              });
+
+              const authorThumbsDown = reactions.data.some(r =>
+                r.user.id === issue.user.id && r.content === '-1'
+              );
+
+              if (authorThumbsDown) {
+                console.log(`  Issue author gave thumbs-down on duplicate comment, skipping`);
+                continue;
+              }
+
+              // Extract the primary duplicate issue number from the comment
+              const dupeMatch = lastDupeComment.body.match(/#(\d+)/);
+              const dupeNumber = dupeMatch ? dupeMatch[1] : 'unknown';
+
+              // Close the issue
+              console.log(`  Closing as duplicate of #${dupeNumber}`);
+
+              await github.rest.issues.update({
+                owner,
+                repo,
+                issue_number: issue.number,
+                state: 'closed',
+                state_reason: 'duplicate',
+              });
+
+              await github.rest.issues.addLabels({
+                owner,
+                repo,
+                issue_number: issue.number,
+                labels: ['autoclose'],
+              });
+
+              await github.rest.issues.createComment({
+                owner,
+                repo,
+                issue_number: issue.number,
+                body: `This issue has been automatically closed as a duplicate of #${dupeNumber}.\n\nIf this is incorrect, please reopen this issue or create a new one.\n\n🤖 Generated with [Claude Code](https://claude.ai/code)`,
+              });
+
+              closedCount++;
+            }
+
+            console.log(`Done. Closed ${closedCount} duplicate issue(s).`);

.github/workflows/claude-dedupe-issues.yml

@@ -0,0 +1,43 @@
+name: Claude Issue Dedupe
+
+on:
+  issues:
+    types: [opened]
+  workflow_dispatch:
+    inputs:
+      issue_number:
+        description: 'Issue number to check for duplicates'
+        required: true
+        type: string
+
+permissions:
+  contents: read
+  issues: write
+  id-token: write
+
+jobs:
+  dedupe:
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Configure AWS Credentials (OIDC)
+        uses: aws-actions/configure-aws-credentials@v4
+        with:
+          role-to-assume: ${{ secrets.BEDROCK_ACCESS_ROLE }}
+          aws-region: us-east-1
+
+      - name: Run duplicate detection
+        uses: anthropics/claude-code-action@v1
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          GITHUB_REPOSITORY: ${{ github.repository }}
+          DUPLICATE_GRACE_DAYS: ${{ vars.DUPLICATE_GRACE_DAYS }}
+        with:
+          use_bedrock: "true"
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          allowed_bots: "github-actions[bot]"
+          prompt: "/dedupe ${{ github.repository }}/issues/${{ github.event.issue.number || inputs.issue_number }}"
+          claude_args: "--model us.anthropic.claude-sonnet-4-5-20250929-v1:0"

.github/workflows/remove-duplicate-on-activity.yml

@@ -0,0 +1,42 @@
+name: Remove duplicate label on activity
+
+on:
+  issue_comment:
+    types: [created]
+
+permissions:
+  issues: write
+
+jobs:
+  remove-duplicate:
+    if: |
+      github.event.issue.state == 'open' &&
+      contains(github.event.issue.labels.*.name, 'duplicate') &&
+      github.event.comment.user.type != 'Bot'
+    runs-on: ubuntu-latest
+    steps:
+      - name: Remove duplicate label
+        uses: actions/github-script@v7
+        with:
+          script: |
+            const { owner, repo } = context.repo;
+            const issueNumber = context.issue.number;
+            const commenter = context.payload.comment.user.login;
+
+            console.log(`Removing duplicate label from issue #${issueNumber} due to comment from ${commenter}`);
+
+            try {
+              await github.rest.issues.removeLabel({
+                owner,
+                repo,
+                issue_number: issueNumber,
+                name: 'duplicate',
+              });
+              console.log(`Successfully removed duplicate label from issue #${issueNumber}`);
+            } catch (error) {
+              if (error.status === 404) {
+                console.log(`duplicate label was already removed from issue #${issueNumber}`);
+              } else {
+                throw error;
+              }
+            }

scripts/comment-on-duplicates.sh

@@ -0,0 +1,90 @@
+#!/bin/bash
+#
+# Copyright OpenSearch Contributors
+# SPDX-License-Identifier: Apache-2.0
+#
+# Posts a formatted duplicate detection comment and adds the duplicate label.
+#
+# Usage:
+#   ./scripts/comment-on-duplicates.sh --base-issue 123 --potential-duplicates 456 789
+
+set -euo pipefail
+
+REPO="${GITHUB_REPOSITORY:-}"
+BASE_ISSUE=""
+DUPLICATES=()
+
+while [[ $# -gt 0 ]]; do
+  case $1 in
+    --base-issue)
+      BASE_ISSUE="$2"
+      shift 2
+      ;;
+    --potential-duplicates)
+      shift
+      while [[ $# -gt 0 && ! "$1" =~ ^-- ]]; do
+        DUPLICATES+=("$1")
+        shift
+      done
+      ;;
+    *)
+      echo "Unknown argument: $1" >&2
+      exit 1
+      ;;
+  esac
+done
+
+if [[ -z "$BASE_ISSUE" ]]; then
+  echo "Error: --base-issue is required" >&2
+  exit 1
+fi
+
+if [[ ${#DUPLICATES[@]} -eq 0 ]]; then
+  echo "Error: --potential-duplicates requires at least one issue number" >&2
+  exit 1
+fi
+
+REPO_FLAG=()
+if [[ -n "$REPO" ]]; then
+  REPO_FLAG=("--repo" "$REPO")
+fi
+
+# Build duplicate list
+DUP_LIST=""
+for dup in "${DUPLICATES[@]}"; do
+  TITLE=$(gh issue view "$dup" "${REPO_FLAG[@]}" --json title -q .title 2>/dev/null || echo "")
+  if [[ -n "$TITLE" ]]; then
+    DUP_LIST+="- #${dup} — ${TITLE}"$'\n'
+  else
+    DUP_LIST+="- #${dup}"$'\n'
+  fi
+done
+
+# Build the comment body with a hidden marker for auto-close detection
+BODY="<!-- duplicate-detection -->
+### Possible Duplicate
+
+Found **${#DUPLICATES[@]}** possible duplicate issue(s):
+
+${DUP_LIST}
+If this is **not** a duplicate:
+- Add a comment on this issue, and the \`duplicate\` label will be removed automatically, or
+- 👎 this comment to prevent auto-closure
+
+Otherwise, this issue will be **automatically closed in ${DUPLICATE_GRACE_DAYS:-7} days**.
+
+🤖 Generated with [Claude Code](https://claude.ai/code)"
+
+# Post the comment
+echo "$BODY" | gh issue comment "$BASE_ISSUE" "${REPO_FLAG[@]}" --body-file -
+
+# Ensure the duplicate label exists
+gh label create "duplicate" \
+  --description "Issue is a duplicate of an existing issue" \
+  --color "cccccc" \
+  "${REPO_FLAG[@]}" 2>/dev/null || true
+
+# Add duplicate label
+gh issue edit "$BASE_ISSUE" "${REPO_FLAG[@]}" --add-label "duplicate"
+
+echo "Posted duplicate comment and added duplicate label to issue #${BASE_ISSUE}"