Add advisory support by Kwstubbs · Pull Request #54 · GitHubSecurityLab/seclab-taskflows

Kwstubbs · 2026-02-13T08:38:05Z

Pass --advisory to run_audit.sh to add advisory support to results.

Copilot

Pull request overview

Adds first-class “security advisory” support to the seclab taskflow ecosystem by introducing a new MCP server/toolbox for fetching repository security advisories, wiring it into audit taskflows, and making it optionally runnable from the audit script.

Changes:

Introduce a new security_advisories MCP server plus toolbox definition.
Add a new audit taskflow to fetch advisories, and update existing audit prompts to incorporate advisories from memcache.
Add a --advisory flag to the audit runner to optionally fetch advisories as part of the pipeline.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
`src/seclab_taskflows/toolboxes/security_advisories.yaml`	Adds toolbox wiring for the new security advisories MCP server.
`src/seclab_taskflows/taskflows/audit/filter_severity.yaml`	Aligns SPDX header format with repo convention.
`src/seclab_taskflows/taskflows/audit/fetch_security_advisories.yaml`	New taskflow to fetch/review advisories (header format currently inconsistent).
`src/seclab_taskflows/taskflows/audit/classify_application_local.yaml`	Updates classification prompt to consider advisories from memcache; adds required toolboxes.
`src/seclab_taskflows/taskflows/audit/audit_issue_local_iter.yaml`	Updates audit prompt to consider advisories from memcache; adds required toolboxes.
`src/seclab_taskflows/personalities/web_application_security_expert.yaml`	Makes the new toolbox available to the security expert personality.
`src/seclab_taskflows/mcp_servers/security_advisories.py`	Implements advisory fetch/filter logic and stores results into memcache.
`scripts/audit/run_audit.sh`	Adds `--advisory` flag to optionally fetch advisories during audits.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

src/seclab_taskflows/taskflows/audit/fetch_security_advisories.yaml

src/seclab_taskflows/mcp_servers/security_advisories.py

….yaml Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

kevinbackhouse · 2026-02-13T14:24:51Z

I think this is duplicating some of the code that's already in ghsa.py

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

src/seclab_taskflows/mcp_servers/security_advisories.py:108

The pagination logic could potentially result in an infinite loop if the API keeps returning exactly 100 results indefinitely. Consider adding a maximum page limit as a safety measure, or tracking if we've seen duplicate results across pages. The existing ghsa.py server handles this more robustly by using the Link header from the API response to determine if there's a next page.

    while True:
        url = f"https://api.github.com/repos/{owner}/{repo}/security-advisories?per_page=100&page={page}"
        headers = {
            "Accept": "application/vnd.github+json",
            "Authorization": f"Bearer {GH_TOKEN}",
            "X-GitHub-Api-Version": "2022-11-28",
        }
        try:
            async with httpx.AsyncClient() as client:
                r = await client.get(url, headers=headers)
                r.raise_for_status()
                data = r.text
                page_advisories = json.loads(data)
                if not page_advisories:
                    break
                advisories.extend(page_advisories)
                if len(page_advisories) < 100:
                    break
                page += 1

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

src/seclab_taskflows/mcp_servers/security_advisories.py

src/seclab_taskflows/taskflows/audit/classify_application_local.yaml

m-y-mo · 2026-02-16T12:37:34Z

src/seclab_taskflows/taskflows/audit/classify_application_local.yaml

-
+        Based on this, determine whether the component is likely to have security problems.
+
+        ## Known Security Advisories for this Repository


Maybe pull this out and turn it into a reusable prompt and include it in this and the other taskflow? Use this version is better because it instructs the LLM to skip the advisory analysis if advisory isn't found.

Sure, makes sense. Let me know if it's what you were thinking.

src/seclab_taskflows/taskflows/audit/audit_issue_local_iter.yaml

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 6 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-17T08:54:47Z

src/seclab_taskflows/taskflows/audit/fetch_security_advisories.yaml

+        - seclab_taskflows.toolboxes.local_file_viewer
+        - seclab_taskflows.toolboxes.gh_file_viewer


fetch_security_advisories is only describing fetching advisories + writing to memcache, but the task grants local_file_viewer and gh_file_viewer toolboxes as well. If they aren’t needed for this flow, removing them reduces tool surface area and avoids unnecessary tool calls/context overhead.

Suggested change

- seclab_taskflows.toolboxes.local_file_viewer

- seclab_taskflows.toolboxes.gh_file_viewer

Copilot · 2026-02-17T08:54:48Z

src/seclab_taskflows/taskflows/audit/fetch_security_advisories.yaml

+      user_prompt: |
+        Fetch all GitHub Security Advisories (GHSAs) for the repo {{ globals.repo }}.
+
+        After fetching, store the list of advisories in memcache under the key 'security_advisories_{{ globals.repo }}'.
+
+        Provide a summary of:
+        1. How many advisories were found
+        2. The severity levels of the advisories
+        3. Key recommendations for addressing them


Downstream prompts expect to be able to reason about “how many advisories were found”, but this taskflow doesn’t define a stable memcache value format (e.g., empty list vs error string like “No advisories found.”). Consider normalizing what gets stored (e.g., always JSON list; store [] on no results; store a structured error separately) so later steps can reliably count/skip advisories.

Copilot · 2026-02-17T08:54:48Z

src/seclab_taskflows/prompts/audit/known_security_advisories.yaml

+prompt: |
+        ## Known Security Advisories for this Repository
+
+        Fetch the security advisories for {{ globals.repo }} from memcache (stored under the key 'security_advisories_{{ globals.repo }}'). If the value in the memcache is null, clearly state so and skip advisory analysis. Otherwise, state how many advisories were found.


This prompt only checks for a null memcache value, but the GHSA fetch tool can also yield non-null sentinel strings (e.g. “No advisories found.”) or error strings. Handle those cases explicitly (treat as empty/skip advisory analysis), otherwise later steps may incorrectly claim advisories exist or fail to report a count.

Suggested change

Fetch the security advisories for {{ globals.repo }} from memcache (stored under the key 'security_advisories_{{ globals.repo }}'). If the value in the memcache is null, clearly state so and skip advisory analysis. Otherwise, state how many advisories were found.

Fetch the security advisories for {{ globals.repo }} from memcache (stored under the key 'security_advisories_{{ globals.repo }}'). If the value in the memcache is null, a sentinel string indicating no advisories (for example, "No advisories found."), or an error message, clearly state that no advisories are available and skip advisory analysis. Otherwise, state how many advisories were found.

Copilot · 2026-02-17T08:54:48Z

src/seclab_taskflows/taskflows/audit/audit_issue_local_iter.yaml

+        {% include 'seclab_taskflows.prompts.audit.known_security_advisories' %}
+
        {% include 'seclab_taskflows.prompts.audit.audit_issue' %}


Including the full “known security advisories” section inside a repeat_prompt audit loop means the same advisories may be fetched/summarized for every issue, increasing token usage and prompt size. Consider fetching/summarizing advisories once (outside the loop) and storing a short summary in memcache, then referencing that summary here.

Copilot · 2026-02-17T08:54:49Z

scripts/audit/run_audit.sh

+    *)
+      echo "Unknown option: $1"
+      exit 1
+      ;;


Unknown option handling prints to stdout and doesn’t show usage/help. Prefer writing option parse errors to stderr and including the usage string (or supporting --help) to make failures easier to diagnose in scripts/CI.

Copilot · 2026-02-17T08:54:49Z

scripts/audit/run_audit_in_docker.sh

 __dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

-${__dir}/../run_in_docker.sh ${__dir}/run_audit.sh "$1"
+${__dir}/../run_in_docker.sh ${__dir}/run_audit.sh "$@"


Consider quoting the script paths when invoking run_in_docker.sh/run_audit.sh (e.g., "${__dir}/../run_in_docker.sh") to avoid issues if the repo path contains spaces.

Suggested change

${__dir}/../run_in_docker.sh ${__dir}/run_audit.sh "$@"

"${__dir}/../run_in_docker.sh" "${__dir}/run_audit.sh" "$@"

m-y-mo · 2026-02-17T09:20:51Z

@anticomputer Is there a way to use the Jinja syntax to make it so that the prompt related to fetching and reviewing advisories is only included when the command line is passed?

anticomputer · 2026-02-19T20:07:44Z

@anticomputer Is there a way to use the Jinja syntax to make it so that the prompt related to fetching and reviewing advisories is only included when the command line is passed?

I'll take a poke now

anticomputer · 2026-02-19T20:19:26Z

@anticomputer Is there a way to use the Jinja syntax to make it so that the prompt related to fetching and reviewing advisories is only included when the command line is passed?

Implemented using the -g logic we use for repo passing as well in 9ec009c cc @m-y-mo

@Kwstubbs can you test to make sure that works for your intended logic still?

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-20T09:07:26Z

src/seclab_taskflows/taskflows/audit/fetch_security_advisories.yaml

+
+        If fetching is successful, store the list of advisories in memcache under the key 'security_advisories_{{ globals.repo }}'.
+
+        If one ore more advisories are found, provide a summary of the findings including:


Typo in line 29: "one ore more" should be "one or more".

Suggested change

If one ore more advisories are found, provide a summary of the findings including:

If one or more advisories are found, provide a summary of the findings including:

memkizz and others added 2 commits February 13, 2026 08:09

Add inital adivsories

a618b14

Pass linter

26f6d3d

Kwstubbs requested a review from m-y-mo as a code owner February 13, 2026 08:38

Copilot AI review requested due to automatic review settings February 13, 2026 08:38

Kwstubbs requested review from JarLob, anticomputer, kevinbackhouse, p- and sylwia-budzynska as code owners February 13, 2026 08:38

Copilot started reviewing on behalf of Kwstubbs February 13, 2026 08:38 View session

Copilot AI reviewed Feb 13, 2026

View reviewed changes

Update src/seclab_taskflows/taskflows/audit/fetch_security_advisories…

ef5f389

….yaml Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings February 13, 2026 14:18

Copilot started reviewing on behalf of kevinbackhouse February 13, 2026 14:19 View session

Copilot AI reviewed Feb 13, 2026

View reviewed changes

src/seclab_taskflows/mcp_servers/security_advisories.py Outdated Show resolved Hide resolved

src/seclab_taskflows/mcp_servers/security_advisories.py Outdated Show resolved Hide resolved

Use ghsa.py

7276a2d

kevinbackhouse previously approved these changes Feb 16, 2026

View reviewed changes

m-y-mo reviewed Feb 16, 2026

View reviewed changes

src/seclab_taskflows/taskflows/audit/classify_application_local.yaml Outdated Show resolved Hide resolved

m-y-mo reviewed Feb 16, 2026

View reviewed changes

src/seclab_taskflows/taskflows/audit/audit_issue_local_iter.yaml Outdated Show resolved Hide resolved

Kwstubbs added 2 commits February 17, 2026 08:00

Add advisory support as a reusable prompt

250186f

add advisory support for run_audit_in_docker.sh

f1ba721

Copilot AI review requested due to automatic review settings February 17, 2026 08:48

Kwstubbs dismissed kevinbackhouse’s stale review via f1ba721 February 17, 2026 08:48

Copilot started reviewing on behalf of Kwstubbs February 17, 2026 08:48 View session

Kwstubbs requested a review from m-y-mo February 17, 2026 08:50

Copilot AI reviewed Feb 17, 2026

View reviewed changes

Make advisory prompt conditional via use_advisory global

9ec009c

Increase verbosity with error message

35e41ee

Copilot AI review requested due to automatic review settings February 20, 2026 09:04

Copilot started reviewing on behalf of Kwstubbs February 20, 2026 09:04 View session

Copilot AI reviewed Feb 20, 2026

View reviewed changes


		Based on this, determine whether the component is likely to have security problems.

		## Known Security Advisories for this Repository

		- seclab_taskflows.toolboxes.local_file_viewer
		- seclab_taskflows.toolboxes.gh_file_viewer

	Fetch the security advisories for {{ globals.repo }} from memcache (stored under the key 'security_advisories_{{ globals.repo }}'). If the value in the memcache is null, clearly state so and skip advisory analysis. Otherwise, state how many advisories were found.
	Fetch the security advisories for {{ globals.repo }} from memcache (stored under the key 'security_advisories_{{ globals.repo }}'). If the value in the memcache is null, a sentinel string indicating no advisories (for example, "No advisories found."), or an error message, clearly state that no advisories are available and skip advisory analysis. Otherwise, state how many advisories were found.

		{% include 'seclab_taskflows.prompts.audit.known_security_advisories' %}

		{% include 'seclab_taskflows.prompts.audit.audit_issue' %}

	${__dir}/../run_in_docker.sh ${__dir}/run_audit.sh "$@"
	"${__dir}/../run_in_docker.sh" "${__dir}/run_audit.sh" "$@"


		If fetching is successful, store the list of advisories in memcache under the key 'security_advisories_{{ globals.repo }}'.

		If one ore more advisories are found, provide a summary of the findings including:

	If one ore more advisories are found, provide a summary of the findings including:
	If one or more advisories are found, provide a summary of the findings including:

Comments

Conversation

Kwstubbs commented Feb 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kevinbackhouse commented Feb 13, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

m-y-mo Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

Kwstubbs Feb 17, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Feb 17, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 17, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 17, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 17, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 17, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 17, 2026

Choose a reason for hiding this comment

Uh oh!

m-y-mo commented Feb 17, 2026

Uh oh!

anticomputer commented Feb 19, 2026

Uh oh!

anticomputer commented Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Kwstubbs commented Feb 13, 2026 •

edited

Loading

anticomputer commented Feb 19, 2026 •

edited

Loading