Enhance immediate-response workflow with Copilot features#3272
Enhance immediate-response workflow with Copilot features#3272
Conversation
Refactor GitHub Actions workflow to improve PR and issue responses with Copilot integration.
|
👋 Hi! Thank you for this contribution! Just to let you know, our GitHub SDK team does a round of issue and PR reviews twice a week, every Monday and Friday! We have a process in place for prioritizing and responding to your input. Because you are a part of this community please feel free to comment, add to, or pick up any issues/PRs that are labeled with |
There was a problem hiding this comment.
Pull request overview
This PR refactors the .github/workflows/immediate-response.yaml GitHub Actions workflow to split PR vs issue automation and add Copilot Models–powered issue triage to generate more actionable, template-aware follow-up comments.
Changes:
- Split the prior combined responder into
respond-to-pr(PR greeting) andtriage-issue(issue triage + response). - Updated PR greeting message to set expectations around Copilot + human triage.
- Added issue triage logic (template completeness checks, duplicate search, release context) and a Copilot Models API call to produce concise follow-up questions.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
Should work without it, but doesn't hurt to include it Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
eyalgal
added
the
Type: Maintenance
austenstone
left a comment
austenstone
left a comment
There was a problem hiding this comment.
Left two comments: one on missing [MAINT] handling in the issue-type parsing, and one small fix on the bot name (github-actions[bot]).
…THUB_TOKEN env - Fix `githubactions[bot]` → `github-actions[bot]` in both jobs (austenstone) - Add `[MAINT]` to issue type detection and cleanTitle regex (austenstone) - Add `env: GITHUB_TOKEN` to github-script step for Models API auth (deiga)
- Detect GitHub's `_No response_` placeholder as a non-answer - Change greeting to "A human maintainer will review this" (drop "from the GitHub SDK team")
There was a problem hiding this comment.
I want to hold off on this PR. I view this as a cool and clever use case for what Copilot can do, but I do not want to experiment on this repository with unproven Copilot uses. This repository is already under-staffed, and potential improvements like this come with a real risk of increasing workload instead of reducing it. For now, let's implement Copilot only when it has proven its value. For example, I view CCR as a proven value.
When coming back to this PR, some non-exhaustive thoughts:
- I don't think this organization has Copilot enabled. That needs to be resolved before implementing this or CCR.
- The logic to review issues should move to a new file, and actions should call that file. That will make it easier to test and to review.
- The model chosen,
gpt-40-mini, should be a variable
|
@robert-crandall I hear the concern about experimenting on an under-staffed repo, but I'd push back a bit on the framing here. The current workflow is actively unhelpful: it promises "reviews twice a week, every Monday and Friday" which hasn't been true in a long time, and gives zero triage value. A maintainer's first touch on every issue is spent asking for basic info that the template already requested. With 40-50 new issues a month, that adds up fast. This isn't an aggressive use of Copilot. It's template validation, duplicate search, and targeted follow-up questions. I simulated it against real issues from this repo, and in every case it asked for exactly what maintainers end up asking manually, sometimes hours later. I'm happy to share those simulations. I'll monitor it closely after merge and we can roll back immediately if it creates noise instead of reducing it. But waiting for "proven value" on a workflow that replaces something already broken feels like the wrong bar. Re: the specific suggestions: extracting the logic into a separate file and making the model a variable are good improvements, but I'd treat those as follow-ups rather than blockers. The workflow works as-is, and for this use case |
eyalgal
requested review from
austenstone,
robert-crandall and
stevehipwell
and removed request for
robert-crandall
stevehipwell
left a comment
stevehipwell
left a comment
There was a problem hiding this comment.
At a high level I agree that the current workflow doesn't really fit but I'm not sure about the value of adding more signal (we already have too much signal). Wouldn't a better MVP solution be to fix the current behaviour? How about an initial PR to remove this workflow and add a separate workflow for issues and PRs just adding a correct comment in the vein of the original. I'm confident that this would be non-contentious.
After there are separate MVP workflows you could re-add these suggestions as 2 seperate PRs which would be easier to review.
| github.event_name == 'pull_request_target' && | ||
| github.actor != 'dependabot[bot]' && | ||
| github.actor != 'renovate[bot]' && | ||
| github.actor != 'github-actions[bot]' && | ||
| github.actor != 'octokitbot' && | ||
| github.repository == 'integrations/terraform-provider-github' |
There was a problem hiding this comment.
Is this a possible combination, isn't github.repository going to point at the source repository?
There was a problem hiding this comment.
@stevehipwell Thanks for the review!
Good catch — you're right that it's redundant here. pull_request_target always runs in the base repo context, and issues only fire on this repo's own tracker. Removed it from both jobs in 7945d60.
I considered splitting into multiple PRs, but the workflow is a single file and the two jobs are independent — splitting them into 3 sequential PRs (remove old → add greeting → add triage) would triple the review cycles without making any individual PR easier to review. I'd rather land this and iterate.
The current workflow actively misleads reporters by promising reviews "twice a week, every Monday and Friday," which hasn't been accurate for a while. This PR replaces that with honest expectations and adds triage that asks for the same missing info maintainers end up requesting manually. If it turns out to add noise instead of reducing it, I'll revert quickly — but leaving the broken workflow in place has a real daily cost.
Would you be willing to convert to an approval given the above? Happy to discuss further.
The `pull_request_target` trigger always runs in the context of the base repo, so `github.repository` is always `integrations/terraform-provider-github` for PRs opened against this repo. For issues, the workflow only fires on the repo's own issue tracker. The check adds no value in either case. Addresses review feedback from @stevehipwell.
Refactor GitHub Actions workflow to improve PR and issue responses with Copilot integration.
This pull request significantly enhances the
.github/workflows/immediate-response.yamlworkflow by splitting and improving the automation for responding to pull requests and issues. It introduces a Copilot-powered triage process for new issues, providing intelligent, actionable feedback to reporters and surfacing potentially related issues to reduce duplicates. The PR also updates the PR response message to set clearer expectations for contributors.Before the change?
Comment with a generic message on new Issues/PRs.
After the change?
Workflow restructuring and messaging improvements:
respondjob into two distinct jobs:respond-to-prfor pull requests andtriage-issuefor issues, each with tailored triggers and permissions.Copilot-powered issue triage automation:
triage-issuejob that uses a GitHub Actions script to:Pull request checklist
Does this introduce a breaking change?
Please see our docs on breaking changes to help!