diff --git a/CHANGELOG.md b/CHANGELOG.md index ebf1b518..9780f21f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,6 +14,15 @@ release cadence. _Changes on `main` since the latest tagged release that have not yet been included in a stable release._ +### Added + +#### VS Code Extension + +- **Built-in portable custom agents** — The extension now ships two `.agent.md` custom agents (`codeql-query-developer`, `codeql-workshop-author`) bundled inside the VSIX. On activation, the extension registers its `agents/` directory in `chat.agentFilesLocations` so both agents are immediately discoverable in VS Code Copilot Chat without any manual configuration. No specific model is required — users choose their own. +- **User-extensibility hooks** — Two new settings allow teams and individuals to extend the bundled agents at runtime (`codeql-mcp.additionalAgentDirs: string[]`) or disable the built-in registration entirely (`codeql-mcp.agents.enabled: boolean`, default `true`). A `--customizations-dir` CLI flag (or `CODEQL_MCP_CUSTOMIZATIONS_DIR` env var) on `bundle:customizations` enables building custom VSIXes with overlay agents, prompts, and skills. +- **`codeql-mcp.showAgentsStatus` command** — New Command Palette entry (**CodeQL MCP: Show Built-in Custom Agents Status**) that reports the enabled state, bundled directory, additional directories, and effective `chat.agentFilesLocations` entries. +- **Bundled prompts and skills** — Four MCP server prompts (`ql-tdd-basic`, `ql-tdd-advanced`, `tools-query-workflow`, `workshop-creation-workflow`) and two skills (`create-codeql-query-development-workshop`, `validate-ql-mcp-server-tools-queries`) are now copied into the VSIX as static contribution points (`chatPromptFiles`, `chatSkills`) so they are available to Copilot Chat without the MCP server running. + ## [v2.25.4] — 2026-05-08 ### Highlights diff --git a/extensions/vscode/.gitignore b/extensions/vscode/.gitignore index a67c0318..29962ee8 100644 --- a/extensions/vscode/.gitignore +++ b/extensions/vscode/.gitignore @@ -4,5 +4,11 @@ server/ coverage/ *.vsix +# Customizations bundle output (generated by scripts/bundle-customizations.js) +agents/ +prompts/ +skills/ +dist-customizations-manifest.json + # @vscode/test-cli downloads and runtime data .vscode-test/ diff --git a/extensions/vscode/.vscodeignore b/extensions/vscode/.vscodeignore index 25d383e8..bc005eab 100644 --- a/extensions/vscode/.vscodeignore +++ b/extensions/vscode/.vscodeignore @@ -27,3 +27,7 @@ coverage/** # Bundled server: exclude test/examples content server/ql/*/tools/test/** server/ql/*/examples/** + +# Customization sources (bundled outputs in agents/, prompts/, skills/ ship in VSIX) +customizations/** +examples/** diff --git a/extensions/vscode/README.md b/extensions/vscode/README.md index 57005f76..59b95a3c 100644 --- a/extensions/vscode/README.md +++ b/extensions/vscode/README.md @@ -33,6 +33,56 @@ On activation (`onStartupFinished`), the extension: 1. **Auto-installs** the `codeql-development-mcp-server` npm package (unless `codeql-mcp.autoInstall` is `false`). 2. **Registers an MCP server definition** (`ql-mcp`) so VS Code's Copilot/MCP integration can discover and launch it. 3. **Watches** the CodeQL extension's storage paths for databases, query results, and MRVA results, passing them to the MCP server as environment variables. +4. **Registers built-in custom agents** (`codeql-query-developer`, `codeql-workshop-author`) in `chat.agentFilesLocations` so they are discoverable in VS Code Copilot Chat. + +## Built-in Custom Agents + +The extension ships two portable `.agent.md` custom agents that appear in VS Code's Copilot Chat agent picker: + +| Agent | Description | +| ----- | ----------- | +| `codeql-query-developer` | Develop CodeQL queries, libraries, and tests using TDD with the `ql-mcp` MCP server tools. | +| `codeql-workshop-author` | Create CodeQL query development workshops from production-grade queries. | + +Both agents use the bundled MCP server tools (`ql-mcp/*`), prompts, and skills that ship with the extension. **No specific model is required** — you choose your own model in VS Code Copilot Chat. + +### Enabling / Disabling the Built-in Agents + +Agents are enabled by default. To disable them, set: + +```json +"codeql-mcp.agents.enabled": false +``` + +This removes the bundled `agents/` directory from `chat.agentFilesLocations` so the agents are no longer discoverable in Copilot Chat. + +### Extending at Runtime — Additional Agent Directories + +To add team or personal `.agent.md` files without rebuilding the extension, use: + +```json +"codeql-mcp.additionalAgentDirs": ["/path/to/your/team-agents"] +``` + +This appends the directory to `chat.agentFilesLocations` alongside the bundled agents. + +### Extending at Build Time — Custom VSIX + +To bundle your own agents, prompts, and skills into a custom VSIX: + +```bash +cd extensions/vscode +npm run bundle:customizations -- --customizations-dir=./examples/team-customizations +npm run package +``` + +Or via environment variable: + +```bash +CODEQL_MCP_CUSTOMIZATIONS_DIR=./examples/team-customizations npm run bundle:customizations +``` + +See [`examples/team-customizations/`](./examples/team-customizations/README.md) for a complete overlay example. ## Configuration @@ -40,6 +90,8 @@ All settings are under the `codeql-mcp` namespace in VS Code settings: | Setting | Default | Description | | ------------------------------------------ | ---------- | ------------------------------------------------------------------- | +| `codeql-mcp.agents.enabled` | `true` | Register the bundled custom agents in `chat.agentFilesLocations`. | +| `codeql-mcp.additionalAgentDirs` | `[]` | Additional `.agent.md` directories appended to `chat.agentFilesLocations`. | | `codeql-mcp.autoInstall` | `true` | Auto-install/update the MCP server on activation. | | `codeql-mcp.serverVersion` | `"latest"` | npm version to install (`"latest"` for most recent). | | `codeql-mcp.serverCommand` | `"node"` | Command to launch the server. Override to `"npx"` or a custom path. | @@ -55,26 +107,28 @@ All settings are under the `codeql-mcp` namespace in VS Code settings: Available from the Command Palette (`Cmd+Shift+P` / `Ctrl+Shift+P`): -| Command | Description | -| ------------------------------------------------- | ----------------------------------------------- | -| **CodeQL MCP: Reinstall MCP Server** | Re-download and install the server package. | -| **CodeQL MCP: Reinstall CodeQL Tool Query Packs** | Re-install the bundled CodeQL tool query packs. | -| **CodeQL MCP: Show Status** | Display current server status. | -| **CodeQL MCP: Show Logs** | Open the server log output. | +| Command | Description | +| -------------------------------------------------------- | ----------------------------------------------- | +| **CodeQL MCP: Reinstall MCP Server** | Re-download and install the server package. | +| **CodeQL MCP: Reinstall CodeQL Tool Query Packs** | Re-install the bundled CodeQL tool query packs. | +| **CodeQL MCP: Show Built-in Custom Agents Status** | Show which agent dirs are registered. | +| **CodeQL MCP: Show Status** | Display current server status. | +| **CodeQL MCP: Show Logs** | Open the server log output. | ## Development ### npm Scripts -| Script | What it does | When to use | -| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------- | -| `npm run package` | **Builds everything and produces the `.vsix`**. Internally runs `vscode:prepublish` (clean → lint → bundle → bundle:server) then `vsce package`. | **Building a distributable `.vsix`.** | -| `npm run build` | `clean` → `lint` → `bundle` (extension only, no server). | Development builds without packaging. | -| `npm run bundle` | esbuild the extension (no lint, no clean). | Fast iteration during development. | -| `npm run watch` | Rebuild the extension on file changes. | Active development. | -| `npm run test` | Run unit tests with Vitest. | Validating changes. | -| `npm run test:coverage` | Run unit tests with coverage. | CI / pre-merge validation. | -| `npm run lint` | Run ESLint on `src/` and `test/`. | Checking code style. | +| Script | What it does | When to use | +| ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------- | +| `npm run package` | **Builds everything and produces the `.vsix`**. Internally runs `vscode:prepublish` (clean → lint → bundle → bundle:server → bundle:customizations) then `vsce package`. | **Building a distributable `.vsix`.** | +| `npm run build` | `clean` → `lint` → `bundle` (extension only, no server or customizations). | Development builds without packaging. | +| `npm run bundle` | esbuild the extension (no lint, no clean). | Fast iteration during development. | +| `npm run bundle:customizations`| Copy bundled agents/prompts/skills to output dirs and write the manifest. | After modifying agent/prompt/skill sources. | +| `npm run watch` | Rebuild the extension on file changes. | Active development. | +| `npm run test` | Run unit tests with Vitest. | Validating changes. | +| `npm run test:coverage` | Run unit tests with coverage. | CI / pre-merge validation. | +| `npm run lint` | Run ESLint on `src/` and `test/`. | Checking code style. | > **Note:** `vscode:prepublish` is a lifecycle hook invoked automatically by `vsce package` — you should not need to run it directly. diff --git a/extensions/vscode/customizations/agents/codeql-query-developer.agent.md b/extensions/vscode/customizations/agents/codeql-query-developer.agent.md new file mode 100644 index 00000000..2ba55e0a --- /dev/null +++ b/extensions/vscode/customizations/agents/codeql-query-developer.agent.md @@ -0,0 +1,59 @@ +--- +name: codeql-query-developer +description: "Develop CodeQL queries, libraries, and tests with TDD via the ql-mcp server." +tools: ['ql-mcp/*', 'edit', 'read', 'search', 'todo'] +--- + +# `codeql-query-developer` Agent + +Develops, tests, and validates CodeQL queries, libraries, and tests using the QL MCP Server tools. + +## Core Capabilities + +- Uses `ql-mcp/*` tools to create and manage CodeQL databases from source code. +- Follows test-driven development (TDD): writes tests with expected results first, then implements queries to pass them. +- Uses `ql-mcp/*` tools to run queries against databases, execute query unit tests, and generate query logs for debugging. +- Organizes queries, libraries, and tests following CodeQL pack conventions (`qlpack.yml`, `codeql-workspace.yml`). +- Documents query purpose, logic, and usage with clear QL comments. +- ALWAYS uses verbose help (`codeql -h -vv`) when learning about `codeql` CLI commands. +- NEVER makes anything up about CodeQL semantics or database schema. +- NEVER assumes query behavior without testing against actual databases. + +## TDD Workflow + +1. **Understand the goal** — clarify what the query should detect and for which language. +2. **Create test code** — write test source files that contain positive and negative examples. +3. **Extract a test database** — use `ql-mcp/codeql_create_database` or `ql-mcp/codeql_query_run` to build a DB. +4. **Write `.qlref` / `.expected` test files** — specify expected results before writing query logic. +5. **Implement the query** — write the `.ql` file to make the tests pass. +6. **Run tests** — use `ql-mcp/codeql_test_run` to execute the unit tests; iterate until 100% pass. +7. **Validate** — run the query against real databases; inspect results; refine as needed. +8. **Document** — add `@name`, `@description`, `@kind`, `@id`, `@tags` metadata to the query. + +## MCP Tool Usage + +Use the bundled `ql-mcp/*` tools for all CodeQL operations: + +- `ql-mcp/codeql_create_database` — create a CodeQL database from source. +- `ql-mcp/codeql_query_run` — run a query against a database. +- `ql-mcp/codeql_test_run` — run CodeQL unit tests. +- `ql-mcp/codeql_query_explain` — explain a query's structure. +- `ql-mcp/find_codeql_query_files` — locate query files in the workspace. +- `ql-mcp/codeql_pack_install` — install QL pack dependencies. + +## Bundled Skills and Prompts + +The following bundled resources are available in the extension and provide detailed step-by-step workflows: + +- **Skill `create-codeql-query-development-workshop`** — reference for structured query development. +- **Skill `validate-ql-mcp-server-tools-queries`** — validate PrintAST, PrintCFG, and CallGraph tools. +- **Prompt `ql-tdd-basic`** — basic TDD workflow for simple CodeQL queries. +- **Prompt `ql-tdd-advanced`** — advanced TDD patterns for data-flow and taint-tracking queries. +- **Prompt `tools-query-workflow`** — workflow for using MCP tool queries (PrintAST, PrintCFG, CallGraph). + +## Quality Standards + +- All solution queries must compile without errors. +- All unit tests must pass at 100%. +- Expected results must be accurate (verified against real test databases). +- Queries must include complete `@name`, `@description`, `@kind`, `@id`, `@tags` metadata. diff --git a/extensions/vscode/customizations/agents/codeql-workshop-author.agent.md b/extensions/vscode/customizations/agents/codeql-workshop-author.agent.md new file mode 100644 index 00000000..94b29936 --- /dev/null +++ b/extensions/vscode/customizations/agents/codeql-workshop-author.agent.md @@ -0,0 +1,74 @@ +--- +name: codeql-workshop-author +description: "Create CodeQL query development workshops from production-grade queries." +tools: ['ql-mcp/*', 'edit', 'read', 'search', 'todo'] +handoffs: + - agent: codeql-query-developer + label: Develop and Test Query + prompt: 'Develop and test a CodeQL query using TDD methodology. Follow the `ql-tdd-basic` or `ql-tdd-advanced` prompt workflow and return the validated query file path and test results when complete.' + send: false +--- + +# `codeql-workshop-author` Agent + +Creates comprehensive CodeQL query development workshops from production-grade queries using the QL MCP Server tools. + +## Core Capabilities + +- Uses `ql-mcp/*` tools to analyze production queries and create workshop materials. +- Follows the bundled `create-codeql-query-development-workshop` skill to generate workshops from production queries. +- Validates AST/CFG tools using the bundled `validate-ql-mcp-server-tools-queries` skill. +- Decomposes queries into 4–8 logical learning stages that guide learners from simple to complex. +- Generates exercise queries, solution queries, unit tests, AST/CFG visualizations, and README documentation. +- ALWAYS uses verbose help (`codeql -h -vv`) when learning about `codeql` CLI commands. +- NEVER makes anything up about CodeQL semantics or database schema. +- NEVER assumes query behavior without testing against actual databases. + +## Workshop Generation Process + +1. **Analyze source query** — use `ql-mcp/find_codeql_query_files` and `ql-mcp/codeql_query_explain` to understand the production query. +2. **Prepare environment** — run `codeql pack install` on solutions and solutions-tests directories; run any `initialize-qltests.sh` scripts. +3. **Validate AST/CFG tools** — use the bundled `validate-ql-mcp-server-tools-queries` skill to confirm PrintAST, PrintCFG, and CallGraph return non-empty output. **Fail if any query returns empty results.** +4. **Plan stages** — decompose the query into 4–8 logical learning stages. +5. **Create workshop structure** — set up directories, `qlpack.yml` files, and `codeql-workspace.yml`. +6. **Generate solution stages** — for each stage, delegate to `codeql-query-developer` (via the **Develop and Test Query** handoff) to create and validate the solution query. +7. **Create exercise queries** — remove implementation details from solutions; add scaffolding, `// TODO` hints, and `select` stubs. +8. **Generate enrichments** — create AST/CFG graphs (from tool output), build scripts, and documentation. +9. **Final validation** — run all solution tests; confirm 100% pass rate before declaring the workshop complete. + +## Workshop Structure + +``` +/ + exercises/ # Student exercise queries (incomplete, with scaffolding) + exercises-tests/ # Unit tests for exercises + solutions/ # Complete solution queries + solutions-tests/ # Unit tests for solutions (must pass 100%) + tests-common/ # Shared test code and databases + graphs/ # AST/CFG visualizations + README.md # Workshop guide + build-databases.sh # Database creation script + codeql-workspace.yml +``` + +## Decomposition Strategies + +- **Syntactic → Semantic** — Start with syntax, add type, control flow, then data flow. +- **Local → Global** — Start local, expand to cross-procedural analysis. +- **Simple → Filtered** — High recall first, then refine with filters. +- **Building Blocks** — Define helpers, combine into sources/sinks, connect with flow. + +## Bundled Skills and Prompts + +- **Skill `create-codeql-query-development-workshop`** — full step-by-step workshop creation workflow. +- **Skill `validate-ql-mcp-server-tools-queries`** — AST/CFG/CallGraph validation protocol. +- **Prompt `workshop-creation-workflow`** — structured prompt for workshop generation from a production query. +- **Prompt `ql-tdd-advanced`** — advanced TDD patterns for data-flow and taint-tracking queries. + +## Quality Standards + +- All solution queries compile without errors. +- All solution tests pass at 100%. +- Exercise queries have appropriate scaffolding (not empty, not complete). +- Expected results progress logically from stage to stage. +- Test code covers positive, negative, and edge cases. diff --git a/extensions/vscode/customizations/bundle-customizations.config.js b/extensions/vscode/customizations/bundle-customizations.config.js new file mode 100644 index 00000000..2976ff70 --- /dev/null +++ b/extensions/vscode/customizations/bundle-customizations.config.js @@ -0,0 +1,23 @@ +/** + * bundle-customizations.config.js + * + * Whitelist of prompts and skills to bundle into the VS Code extension. + * Prompts are sourced from server/src/prompts/. + * Skills are sourced from .github/skills//SKILL.md. + * + * Missing entries are silently skipped with a console.warn — the build + * never fails due to absent optional files. + */ + +export const prompts = [ + 'ql-tdd-basic.prompt.md', + 'ql-tdd-advanced.prompt.md', + 'tools-query-workflow.prompt.md', + 'workshop-creation-workflow.prompt.md', +]; + +export const skills = [ + 'create-codeql-query-development-workshop', + 'create-codeql-query-tdd-generic', + 'validate-ql-mcp-server-tools-queries', +]; diff --git a/extensions/vscode/esbuild.config.js b/extensions/vscode/esbuild.config.js index bd95c587..3f35e150 100644 --- a/extensions/vscode/esbuild.config.js +++ b/extensions/vscode/esbuild.config.js @@ -34,6 +34,7 @@ const testSuiteConfig = { ...shared, entryPoints: [ 'test/suite/index.ts', + 'test/suite/agents.integration.test.ts', 'test/suite/bridge.integration.test.ts', 'test/suite/copydb-e2e.integration.test.ts', 'test/suite/extension.integration.test.ts', diff --git a/extensions/vscode/eslint.config.mjs b/extensions/vscode/eslint.config.mjs index 5a698bd2..202a0e4e 100644 --- a/extensions/vscode/eslint.config.mjs +++ b/extensions/vscode/eslint.config.mjs @@ -88,6 +88,8 @@ export default [ ecmaVersion: 2022, sourceType: 'module', globals: { + clearTimeout: 'readonly', + setTimeout: 'readonly', suite: 'readonly', test: 'readonly', setup: 'readonly', diff --git a/extensions/vscode/examples/team-customizations/README.md b/extensions/vscode/examples/team-customizations/README.md new file mode 100644 index 00000000..2726773d --- /dev/null +++ b/extensions/vscode/examples/team-customizations/README.md @@ -0,0 +1,42 @@ +# Team Customizations — Overlay Example + +This directory demonstrates how to extend the built-in custom agents with +team-specific or personal agents, prompts, and skills. + +## Structure + +``` +team-customizations/ + agents/ # .agent.md files to add or override + prompts/ # .prompt.md files to add + skills/ # /SKILL.md files to add + README.md # this file +``` + +## Usage + +### At build time (produces a custom VSIX) + +```bash +cd extensions/vscode +npm run bundle:customizations -- --customizations-dir=./examples/team-customizations +npm run package +``` + +### At bundle time only (during development) + +```bash +CODEQL_MCP_CUSTOMIZATIONS_DIR=./examples/team-customizations npm run bundle:customizations +``` + +## Override vs. Add + +- A file with the **same name** as a bundled file **replaces** it (with a warning). +- A file with a **new name** is **added** alongside the defaults. + +## Important Notes + +- **Never add a `model:` key** to `.agent.md` files — users choose their own model in VS Code. +- Skill files must be named `SKILL.md` and placed in `skills//SKILL.md`. +- Prompt files must end in `.prompt.md`. +- Agent files must end in `.agent.md`. diff --git a/extensions/vscode/examples/team-customizations/agents/example-override.agent.md b/extensions/vscode/examples/team-customizations/agents/example-override.agent.md new file mode 100644 index 00000000..efb42738 --- /dev/null +++ b/extensions/vscode/examples/team-customizations/agents/example-override.agent.md @@ -0,0 +1,28 @@ +--- +name: example-override +description: "Example agent demonstrating the overlay/override pattern." +tools: ['ql-mcp/*', 'edit', 'read', 'search'] +--- + +# `example-override` Agent + +This is a stub agent that demonstrates how to add or override `.agent.md` files +using the `--customizations-dir` overlay feature. + +## How Overlays Work + +When you run: + +```bash +npm run bundle:customizations -- --customizations-dir=./examples/team-customizations +``` + +Files in `/agents/` are copied **after** the defaults, +replacing any file with the same name and adding new files. + +## Customization Tips + +- Override `codeql-query-developer.agent.md` to add team-specific workflows. +- Add new `.agent.md` files for team-specific roles (e.g., `security-review.agent.md`). +- Keep `name:` in frontmatter matching the filename stem for VS Code to discover the agent. +- Never add a `model:` key — users choose their own model. diff --git a/extensions/vscode/examples/team-customizations/prompts/example-team.prompt.md b/extensions/vscode/examples/team-customizations/prompts/example-team.prompt.md new file mode 100644 index 00000000..0ced5d08 --- /dev/null +++ b/extensions/vscode/examples/team-customizations/prompts/example-team.prompt.md @@ -0,0 +1,10 @@ +--- +name: example-team-prompt +description: An example team prompt demonstrating the overlay feature. +--- + +# Example Team Prompt + +This prompt demonstrates adding a team-specific workflow prompt via the overlay. + +Use it as a template to add your own prompts to the bundled extension. diff --git a/extensions/vscode/examples/team-customizations/skills/example-team-skill/SKILL.md b/extensions/vscode/examples/team-customizations/skills/example-team-skill/SKILL.md new file mode 100644 index 00000000..a492a55d --- /dev/null +++ b/extensions/vscode/examples/team-customizations/skills/example-team-skill/SKILL.md @@ -0,0 +1,19 @@ +# Example Team Skill + +This is an example `SKILL.md` file demonstrating how to add a team-specific skill +via the `--customizations-dir` overlay feature. + +## How to Use + +1. Copy this directory to your team's customizations repository. +2. Rename `example-team-skill` to match your skill's name. +3. Replace this content with your team's skill documentation. +4. Reference it in your `.agent.md` files. + +## Overlay Usage + +```bash +npm run bundle:customizations -- --customizations-dir=./examples/team-customizations +``` + +Skills placed in `/skills//SKILL.md` are merged alongside the defaults. diff --git a/extensions/vscode/package.json b/extensions/vscode/package.json index 4d181f54..75ad610b 100644 --- a/extensions/vscode/package.json +++ b/extensions/vscode/package.json @@ -48,6 +48,14 @@ "configuration": { "title": "CodeQL MCP Server", "properties": { + "codeql-mcp.additionalAgentDirs": { + "type": "array", + "items": { + "type": "string" + }, + "default": [], + "description": "Additional directories containing `.agent.md` files to append to `chat.agentFilesLocations`. Useful for team or personal agent overlays." + }, "codeql-mcp.additionalDatabaseDirs": { "type": "array", "items": { @@ -80,6 +88,11 @@ "default": [], "description": "Additional directories containing query run result subdirectories. Appended to CODEQL_QUERY_RUN_RESULTS_DIRS alongside the vscode-codeql query storage path." }, + "codeql-mcp.agents.enabled": { + "type": "boolean", + "default": true, + "description": "Register the bundled custom agents (`codeql-query-developer`, `codeql-workshop-author`) by appending the extension's bundled `agents/` directory to `chat.agentFilesLocations`. Toggle off to disable." + }, "codeql-mcp.autoDownloadPacks": { "type": "boolean", "default": true, @@ -133,6 +146,38 @@ } } }, + "chatAgents": [ + { + "path": "./agents/codeql-query-developer.agent.md" + }, + { + "path": "./agents/codeql-workshop-author.agent.md" + } + ], + "chatPromptFiles": [ + { + "path": "./prompts/ql-tdd-advanced.prompt.md" + }, + { + "path": "./prompts/ql-tdd-basic.prompt.md" + }, + { + "path": "./prompts/tools-query-workflow.prompt.md" + }, + { + "path": "./prompts/workshop-creation-workflow.prompt.md" + } + ], + "chatSkills": [ + { + "name": "create-codeql-query-development-workshop", + "path": "./skills/create-codeql-query-development-workshop/SKILL.md" + }, + { + "name": "validate-ql-mcp-server-tools-queries", + "path": "./skills/validate-ql-mcp-server-tools-queries/SKILL.md" + } + ], "commands": [ { "command": "codeql-mcp.reinstallServer", @@ -144,6 +189,11 @@ "title": "Reinstall CodeQL Tool Query Packs", "category": "CodeQL MCP" }, + { + "command": "codeql-mcp.showAgentsStatus", + "title": "Show Built-in Custom Agents Status", + "category": "CodeQL MCP" + }, { "command": "codeql-mcp.showStatus", "title": "Show Status", @@ -159,8 +209,9 @@ "scripts": { "build": "npm run clean && npm run lint && npm run bundle", "bundle": "npm run rebuild:esbuild && node esbuild.config.js", + "bundle:customizations": "node scripts/bundle-customizations.js", "bundle:server": "node scripts/bundle-server.js", - "clean": "rm -rf dist server .vscode-test/* *.vsix", + "clean": "rm -rf dist server agents prompts skills .vscode-test/* *.vsix dist-customizations-manifest.json", "download:vscode": "node scripts/download-vscode.js", "lint": "eslint src/ test/", "lint:fix": "eslint src/ test/ --fix", @@ -171,7 +222,7 @@ "test:integration": "npm run download:vscode && vscode-test", "test:integration:label": "vscode-test --label", "test:watch": "vitest --watch", - "vscode:prepublish": "npm run clean && npm run lint && npm run bundle && npm run bundle:server", + "vscode:prepublish": "npm run clean && npm run lint && npm run bundle && npm run bundle:server && npm run bundle:customizations", "watch": "npm run rebuild:esbuild && node esbuild.config.js --watch" }, "devDependencies": { diff --git a/extensions/vscode/scripts/bundle-customizations.js b/extensions/vscode/scripts/bundle-customizations.js new file mode 100644 index 00000000..2600993f --- /dev/null +++ b/extensions/vscode/scripts/bundle-customizations.js @@ -0,0 +1,193 @@ +/** + * bundle-customizations.js + * + * Copies bundled custom agents, whitelisted prompts, and whitelisted skills + * into the extension's output directories so the VSIX is self-contained. + * + * Run via: npm run bundle:customizations + * Called automatically by: vscode:prepublish + * + * Resulting layout inside extensions/vscode/: + * agents/ (bundled .agent.md files) + * prompts/ (whitelisted prompt files) + * skills//SKILL.md (whitelisted skill files) + * dist-customizations-manifest.json (manifest of bundled files) + * + * Overlay support: + * --customizations-dir= or CODEQL_MCP_CUSTOMIZATIONS_DIR= + * After the defaults are copied, files from /{agents,prompts,skills} + * are merged in, replacing any colliding files with a warning. + */ + +import { + copyFileSync, + existsSync, + mkdirSync, + readdirSync, + rmSync, + writeFileSync, +} from 'fs'; +import { basename, dirname, join, normalize, resolve } from 'path'; +import { fileURLToPath } from 'url'; + +const __dirname = dirname(fileURLToPath(import.meta.url)); + +/** + * Core bundle function — separated so tests can call it directly. + * @param {object} opts + * @param {string} opts.extensionRoot Absolute path to extensions/vscode/ + * @param {string|undefined} opts.customizationsDir Optional overlay directory + */ +export async function runBundle({ extensionRoot, customizationsDir }) { + const repoRoot = resolve(extensionRoot, '..', '..'); + const serverPromptsDir = join(repoRoot, 'server', 'src', 'prompts'); + const skillsRoot = join(repoRoot, '.github', 'skills'); + const customizationsSourceDir = join(extensionRoot, 'customizations'); + + const targetAgentsDir = join(extensionRoot, 'agents'); + const targetPromptsDir = join(extensionRoot, 'prompts'); + const targetSkillsDir = join(extensionRoot, 'skills'); + + // Load whitelist config + const configPath = join(customizationsSourceDir, 'bundle-customizations.config.js'); + const { prompts: promptWhitelist, skills: skillWhitelist } = await import(configPath); + + // Clean previous outputs + for (const dir of [targetAgentsDir, targetPromptsDir, targetSkillsDir]) { + if (existsSync(dir)) { + rmSync(dir, { recursive: true, force: true }); + } + mkdirSync(dir, { recursive: true }); + } + + // Track bundled files for manifest + const manifest = { agents: [], prompts: [], skills: [] }; + + // --- Copy agents --- + const agentsSourceDir = join(customizationsSourceDir, 'agents'); + if (existsSync(agentsSourceDir)) { + for (const file of readdirSync(agentsSourceDir)) { + if (!file.endsWith('.agent.md')) continue; + const src = join(agentsSourceDir, file); + const dst = join(targetAgentsDir, file); + copyFileSync(src, dst); + manifest.agents.push(`agents/${file}`); + console.log(`✅ Copied agent: ${file}`); + } + } + + // --- Copy whitelisted prompts --- + for (const promptName of promptWhitelist) { + const src = join(serverPromptsDir, promptName); + if (!existsSync(src)) { + console.warn(`⚠️ Prompt not found, skipping: ${promptName}`); + continue; + } + const dst = join(targetPromptsDir, promptName); + copyFileSync(src, dst); + manifest.prompts.push(`prompts/${promptName}`); + console.log(`✅ Copied prompt: ${promptName}`); + } + + // --- Copy whitelisted skills --- + for (const skillName of skillWhitelist) { + const src = join(skillsRoot, skillName, 'SKILL.md'); + if (!existsSync(src)) { + console.warn(`⚠️ Skill not found, skipping: ${skillName}/SKILL.md`); + continue; + } + const dstDir = join(targetSkillsDir, skillName); + mkdirSync(dstDir, { recursive: true }); + const dst = join(dstDir, 'SKILL.md'); + copyFileSync(src, dst); + manifest.skills.push(`skills/${skillName}/SKILL.md`); + console.log(`✅ Copied skill: ${skillName}/SKILL.md`); + } + + // --- Apply overlay (if specified) --- + if (customizationsDir) { + const overlayRoot = resolve(customizationsDir); + console.log(`\n🔀 Applying overlay from: ${overlayRoot}`); + + for (const category of ['agents', 'prompts', 'skills']) { + const overlayDir = join(overlayRoot, category); + if (!existsSync(overlayDir)) continue; + + applyOverlayDir(overlayDir, join(extensionRoot, category), category, manifest); + } + } + + // --- Write manifest --- + const manifestPath = join(extensionRoot, 'dist-customizations-manifest.json'); + writeFileSync(manifestPath, JSON.stringify(manifest, null, 2) + '\n', 'utf8'); + console.log(`\n📋 Manifest written: dist-customizations-manifest.json`); + + console.log(''); + console.log('🎉 Customizations bundle complete.'); + console.log(` Agents : ${manifest.agents.length}`); + console.log(` Prompts: ${manifest.prompts.length}`); + console.log(` Skills : ${manifest.skills.length}`); + + return manifest; +} + +/** + * Recursively copies files from overlayDir into targetDir. + * Warns when a file already exists (collision). + * + * subPath tracks the path relative to the category root so the manifest key + * preserves the directory structure (e.g. "skills/foo/SKILL.md", not + * "skills/SKILL.md"). + */ +function applyOverlayDir(overlayDir, targetDir, categoryKey, manifest, subPath = '') { + for (const entry of readdirSync(overlayDir, { withFileTypes: true })) { + const srcPath = join(overlayDir, entry.name); + const dstPath = join(targetDir, entry.name); + const nextSubPath = subPath ? `${subPath}/${entry.name}` : entry.name; + + if (entry.isDirectory()) { + mkdirSync(dstPath, { recursive: true }); + applyOverlayDir(srcPath, dstPath, categoryKey, manifest, nextSubPath); + continue; + } + + const alreadyExists = existsSync(dstPath); + if (alreadyExists) { + console.warn(`⚠️ Overlay replaces bundled file: ${normalize(dstPath)}`); + } + + mkdirSync(dirname(dstPath), { recursive: true }); + copyFileSync(srcPath, dstPath); + + // Build a relative manifest key: e.g. "agents/foo.agent.md" or + // "skills/foo/SKILL.md" — preserving any subdirectory structure. + const relKey = `${categoryKey}/${nextSubPath}`; + if (!alreadyExists && Array.isArray(manifest[categoryKey])) { + manifest[categoryKey].push(relKey); + } + + console.log(` ${alreadyExists ? '↩️ Replaced' : '➕ Added'}: ${relKey}`); + } +} + +// --- CLI entry point --- +if (import.meta.url === `file://${process.argv[1]}`) { + // Parse --customizations-dir=PATH or --customizations-dir PATH + let customizationsDir = + process.env.CODEQL_MCP_CUSTOMIZATIONS_DIR ?? undefined; + + for (let i = 2; i < process.argv.length; i++) { + const arg = process.argv[i]; + if (arg.startsWith('--customizations-dir=')) { + customizationsDir = arg.slice('--customizations-dir='.length); + } else if (arg === '--customizations-dir' && process.argv[i + 1]) { + customizationsDir = process.argv[++i]; + } + } + + const extensionRoot = resolve(__dirname, '..'); + runBundle({ extensionRoot, customizationsDir }).catch((err) => { + console.error('❌ bundle-customizations failed:', err); + process.exit(1); + }); +} diff --git a/extensions/vscode/src/customizations/agent-registrar.ts b/extensions/vscode/src/customizations/agent-registrar.ts new file mode 100644 index 00000000..09d23b2a --- /dev/null +++ b/extensions/vscode/src/customizations/agent-registrar.ts @@ -0,0 +1,183 @@ +import * as path from 'path'; +import * as vscode from 'vscode'; +import { Logger } from '../common/logger'; + +/** Normalizes a file-system path for comparison (removes trailing sep, platform lower-case on Windows). */ +function normalizePath(p: string): string { + let n = path.normalize(p); + // Remove trailing separator + if (n.endsWith(path.sep) && n.length > 1) { + n = n.slice(0, -1); + } + return process.platform === 'win32' ? n.toLowerCase() : n; +} + +/** + * Manages registration of `.agent.md` directories in `chat.agentFilesLocations`. + * + * Adds the extension's bundled `agents/` directory and any user-configured + * additional directories to the VS Code configuration setting, and removes + * them on dispose. + */ +export class AgentRegistrar implements vscode.Disposable { + private readonly _bundledDir: string; + private readonly _disposables: vscode.Disposable[] = []; + private _addedKeys: Set = new Set(); + + constructor( + private readonly _context: vscode.ExtensionContext, + private readonly _logger: Logger, + ) { + this._bundledDir = path.join(_context.extensionUri.fsPath, 'agents'); + } + + /** Registers bundled + additional agent dirs in chat.agentFilesLocations. */ + register(): void { + const cfg = vscode.workspace.getConfiguration('codeql-mcp'); + const enabled = cfg.get('agents.enabled', true); + + if (!enabled) { + this._removeAddedKeys(); + return; + } + + const additionalDirs = cfg.get('additionalAgentDirs', []); + + const dirsToAdd = [this._bundledDir, ...additionalDirs]; + + const chatCfg = vscode.workspace.getConfiguration('chat'); + const existing: Record = + this._normalizeLocations(chatCfg.get('agentFilesLocations')); + + const updated = { ...existing }; + const newKeys: Set = new Set(); + + for (const dir of dirsToAdd) { + const normalized = normalizePath(dir); + if (!this._hasNormalizedKey(updated, normalized)) { + updated[dir] = true; + newKeys.add(dir); + } + } + + if (Object.keys(updated).length === Object.keys(existing).length) { + // Nothing new to add; make sure we track the keys we added previously + this._addedKeys = newKeys; + return; + } + + const target = this._configTarget(); + chatCfg.update('agentFilesLocations', updated, target).then( + () => { + this._addedKeys = newKeys; + this._logger.info( + `AgentRegistrar: registered ${newKeys.size} dir(s) in chat.agentFilesLocations`, + ); + }, + (err: unknown) => { + this._logger.warn( + `AgentRegistrar: failed to update chat.agentFilesLocations: ${err instanceof Error ? err.message : String(err)}`, + ); + }, + ); + } + + /** Subscribes to configuration and workspace folder changes. */ + startWatching(): void { + this._disposables.push( + vscode.workspace.onDidChangeConfiguration((e) => { + if ( + e.affectsConfiguration('codeql-mcp.agents') || + e.affectsConfiguration('codeql-mcp.additionalAgentDirs') + ) { + this.register(); + } + }), + vscode.workspace.onDidChangeWorkspaceFolders(() => { + this.register(); + }), + ); + } + + /** Returns current status for the showAgentsStatus command. */ + getStatus(): { + enabled: boolean; + bundledDir: string; + additionalDirs: string[]; + effectiveLocations: string[]; + } { + const cfg = vscode.workspace.getConfiguration('codeql-mcp'); + const enabled = cfg.get('agents.enabled', true); + const additionalDirs = cfg.get('additionalAgentDirs', []); + + const chatCfg = vscode.workspace.getConfiguration('chat'); + const locations = this._normalizeLocations(chatCfg.get('agentFilesLocations')); + + return { + additionalDirs, + bundledDir: this._bundledDir, + effectiveLocations: Object.keys(locations), + enabled, + }; + } + + dispose(): void { + this._removeAddedKeys(); + for (const d of this._disposables) { + d.dispose(); + } + this._disposables.length = 0; + } + + // ---------- private helpers ---------- + + private _normalizeLocations(value: unknown): Record { + if (!value || typeof value !== 'object') return {}; + if (Array.isArray(value)) { + const obj: Record = {}; + for (const entry of value) { + if (typeof entry === 'string') obj[entry] = true; + } + return obj; + } + return value as Record; + } + + private _hasNormalizedKey(obj: Record, normalized: string): boolean { + return Object.keys(obj).some((k) => normalizePath(k) === normalized); + } + + private _removeAddedKeys(): void { + if (this._addedKeys.size === 0) return; + + const chatCfg = vscode.workspace.getConfiguration('chat'); + const existing = this._normalizeLocations(chatCfg.get('agentFilesLocations')); + const updated: Record = {}; + + for (const [k, v] of Object.entries(existing)) { + if (!this._addedKeys.has(k)) { + updated[k] = v; + } + } + + const target = this._configTarget(); + chatCfg.update('agentFilesLocations', updated, target).then( + () => { + this._logger.info('AgentRegistrar: removed registered agent dirs from chat.agentFilesLocations'); + this._addedKeys = new Set(); + }, + (err: unknown) => { + this._logger.warn( + `AgentRegistrar: failed to clean up chat.agentFilesLocations: ${err instanceof Error ? err.message : String(err)}`, + ); + }, + ); + } + + private _configTarget(): vscode.ConfigurationTarget { + const folders = vscode.workspace.workspaceFolders; + return folders && folders.length > 0 + ? vscode.ConfigurationTarget.Workspace + : vscode.ConfigurationTarget.Global; + } +} diff --git a/extensions/vscode/src/extension.ts b/extensions/vscode/src/extension.ts index db6c2aa6..bbed6d48 100644 --- a/extensions/vscode/src/extension.ts +++ b/extensions/vscode/src/extension.ts @@ -1,4 +1,5 @@ import * as vscode from 'vscode'; +import { AgentRegistrar } from './customizations/agent-registrar'; import { Logger } from './common/logger'; import { CliResolver } from './codeql/cli-resolver'; import { ServerManager } from './server/server-manager'; @@ -38,6 +39,18 @@ export async function activate( disposables.push(cliResolver, serverManager, packInstaller, storagePaths, envBuilder, mcpProvider); + // --- Agent registrar --- + const agentRegistrar = new AgentRegistrar(context, logger); + disposables.push(agentRegistrar); + try { + agentRegistrar.register(); + agentRegistrar.startWatching(); + } catch (err) { + logger.warn( + `AgentRegistrar init failed: ${err instanceof Error ? err.message : String(err)}`, + ); + } + // --- Bridge: filesystem watchers --- const config = vscode.workspace.getConfiguration('codeql-mcp'); const watchEnabled = config.get('watchCodeqlExtension', true); @@ -140,6 +153,21 @@ export async function activate( ); vscode.window.showInformationMessage('CodeQL tool query packs reinstalled successfully.'); }), + vscode.commands.registerCommand('codeql-mcp.showAgentsStatus', () => { + const status = agentRegistrar.getStatus(); + const lines = [ + `Enabled: ${status.enabled}`, + `Bundled agents dir: ${status.bundledDir}`, + `Additional dirs: ${status.additionalDirs.length > 0 ? status.additionalDirs.join(', ') : '(none)'}`, + `Registered locations: ${status.effectiveLocations.length}`, + ]; + logger.info('--- Agents Status ---'); + for (const line of lines) { + logger.info(line); + } + logger.show(); + vscode.window.showInformationMessage(lines.join(' | ')); + }), vscode.commands.registerCommand('codeql-mcp.showStatus', async () => { const cliPath = await cliResolver.resolve(); const version = await serverManager.getInstalledVersion(); diff --git a/extensions/vscode/test/customizations/agent-registrar.test.ts b/extensions/vscode/test/customizations/agent-registrar.test.ts new file mode 100644 index 00000000..381c7881 --- /dev/null +++ b/extensions/vscode/test/customizations/agent-registrar.test.ts @@ -0,0 +1,225 @@ +/** + * Tests for AgentRegistrar. + * + * Uses the existing __mocks__/vscode.ts via vitest.config.ts resolve.alias. + */ + +import { describe, it, expect, vi, beforeEach } from 'vitest'; +import * as vscode from 'vscode'; + +// We need a stateful config mock so we can verify updates +function createStatefulConfigMock(initialValues: Record = {}) { + const store: Record = { ...initialValues }; + return { + get: vi.fn((key: string, defaultVal?: unknown) => { + return key in store ? store[key] : defaultVal; + }), + has: vi.fn(() => false), + inspect: vi.fn(() => undefined), + update: vi.fn((key: string, value: unknown) => { + store[key] = value; + return Promise.resolve(); + }), + _store: store, + }; +} + +// Module-level config stores so we can inspect/update them in tests +let codeqlMcpCfg = createStatefulConfigMock({ 'agents.enabled': true, 'additionalAgentDirs': [] }); +let chatCfg = createStatefulConfigMock({ 'agentFilesLocations': {} }); + +vi.spyOn(vscode.workspace, 'getConfiguration').mockImplementation((section?: string) => { + if (section === 'codeql-mcp') return codeqlMcpCfg as any; + if (section === 'chat') return chatCfg as any; + return createStatefulConfigMock() as any; +}); + +// Set workspaceFolders to a non-empty array for Workspace target tests +vi.spyOn(vscode.workspace, 'workspaceFolders', 'get').mockReturnValue([{ uri: { fsPath: '/ws' } }] as any); + +import { AgentRegistrar } from '../../src/customizations/agent-registrar'; + +function createMockContext(extensionPath = '/mock/extension'): vscode.ExtensionContext { + return { + extensionUri: { fsPath: extensionPath }, + extensionPath, + subscriptions: [], + } as unknown as vscode.ExtensionContext; +} + +function createMockLogger() { + return { + info: vi.fn(), + warn: vi.fn(), + error: vi.fn(), + debug: vi.fn(), + show: vi.fn(), + dispose: vi.fn(), + } as any; +} + +describe('AgentRegistrar', () => { + beforeEach(() => { + codeqlMcpCfg = createStatefulConfigMock({ 'agents.enabled': true, 'additionalAgentDirs': [] }); + chatCfg = createStatefulConfigMock({ 'agentFilesLocations': {} }); + vi.spyOn(vscode.workspace, 'getConfiguration').mockImplementation((section?: string) => { + if (section === 'codeql-mcp') return codeqlMcpCfg as any; + if (section === 'chat') return chatCfg as any; + return createStatefulConfigMock() as any; + }); + }); + + it('adds bundled agents/ dir to chat.agentFilesLocations when enabled', async () => { + const ctx = createMockContext('/mock/extension'); + const logger = createMockLogger(); + const registrar = new AgentRegistrar(ctx, logger); + registrar.register(); + // Wait for the async update to resolve + await Promise.resolve(); + + const updated = chatCfg._store['agentFilesLocations'] as Record; + const bundledDir = '/mock/extension/agents'; + expect(Object.keys(updated)).toContain(bundledDir); + }); + + it('does not add bundled dir when agents.enabled = false', async () => { + codeqlMcpCfg = createStatefulConfigMock({ 'agents.enabled': false, 'additionalAgentDirs': [] }); + vi.spyOn(vscode.workspace, 'getConfiguration').mockImplementation((section?: string) => { + if (section === 'codeql-mcp') return codeqlMcpCfg as any; + if (section === 'chat') return chatCfg as any; + return createStatefulConfigMock() as any; + }); + + const ctx = createMockContext('/mock/extension'); + const logger = createMockLogger(); + const registrar = new AgentRegistrar(ctx, logger); + registrar.register(); + await Promise.resolve(); + + const updated = chatCfg._store['agentFilesLocations'] as Record | undefined; + expect(updated ? Object.keys(updated) : []).not.toContain('/mock/extension/agents'); + }); + + it('appends additionalAgentDirs alongside bundled dir', async () => { + codeqlMcpCfg = createStatefulConfigMock({ + 'agents.enabled': true, + 'additionalAgentDirs': ['/custom/agents'], + }); + vi.spyOn(vscode.workspace, 'getConfiguration').mockImplementation((section?: string) => { + if (section === 'codeql-mcp') return codeqlMcpCfg as any; + if (section === 'chat') return chatCfg as any; + return createStatefulConfigMock() as any; + }); + + const ctx = createMockContext('/mock/extension'); + const logger = createMockLogger(); + const registrar = new AgentRegistrar(ctx, logger); + registrar.register(); + await Promise.resolve(); + + const updated = chatCfg._store['agentFilesLocations'] as Record; + expect(Object.keys(updated)).toContain('/mock/extension/agents'); + expect(Object.keys(updated)).toContain('/custom/agents'); + }); + + it('does not add duplicate entries on repeated register() calls', async () => { + const ctx = createMockContext('/mock/extension'); + const logger = createMockLogger(); + const registrar = new AgentRegistrar(ctx, logger); + registrar.register(); + await Promise.resolve(); + registrar.register(); + await Promise.resolve(); + + const updated = chatCfg._store['agentFilesLocations'] as Record; + const bundledDir = '/mock/extension/agents'; + const count = Object.keys(updated).filter((k) => k === bundledDir).length; + expect(count).toBe(1); + }); + + it('uses ConfigurationTarget.Workspace when workspaceFolders is non-empty', async () => { + const ctx = createMockContext('/mock/extension'); + const logger = createMockLogger(); + const registrar = new AgentRegistrar(ctx, logger); + registrar.register(); + await Promise.resolve(); + + expect(chatCfg.update).toHaveBeenCalledWith( + 'agentFilesLocations', + expect.any(Object), + vscode.ConfigurationTarget.Workspace, + ); + }); + + it('uses ConfigurationTarget.Global when workspaceFolders is empty', async () => { + vi.spyOn(vscode.workspace, 'workspaceFolders', 'get').mockReturnValueOnce([] as any); + + const ctx = createMockContext('/mock/extension'); + const logger = createMockLogger(); + const registrar = new AgentRegistrar(ctx, logger); + registrar.register(); + await Promise.resolve(); + + expect(chatCfg.update).toHaveBeenCalledWith( + 'agentFilesLocations', + expect.any(Object), + vscode.ConfigurationTarget.Global, + ); + }); + + it('dispose() removes only the entries this instance added', async () => { + // Pre-populate with an existing entry + chatCfg = createStatefulConfigMock({ + 'agentFilesLocations': { '/pre-existing/agents': true }, + }); + vi.spyOn(vscode.workspace, 'getConfiguration').mockImplementation((section?: string) => { + if (section === 'codeql-mcp') return codeqlMcpCfg as any; + if (section === 'chat') return chatCfg as any; + return createStatefulConfigMock() as any; + }); + + const ctx = createMockContext('/mock/extension'); + const logger = createMockLogger(); + const registrar = new AgentRegistrar(ctx, logger); + registrar.register(); + await Promise.resolve(); + + // Both entries should be present + let locs = chatCfg._store['agentFilesLocations'] as Record; + expect(Object.keys(locs)).toContain('/pre-existing/agents'); + expect(Object.keys(locs)).toContain('/mock/extension/agents'); + + registrar.dispose(); + await Promise.resolve(); + + locs = chatCfg._store['agentFilesLocations'] as Record; + expect(Object.keys(locs)).toContain('/pre-existing/agents'); + expect(Object.keys(locs)).not.toContain('/mock/extension/agents'); + }); + + it('dispose() is idempotent — safe to call twice', async () => { + const ctx = createMockContext('/mock/extension'); + const logger = createMockLogger(); + const registrar = new AgentRegistrar(ctx, logger); + registrar.register(); + await Promise.resolve(); + + expect(() => { + registrar.dispose(); + registrar.dispose(); + }).not.toThrow(); + }); + + it('getStatus() returns correct shape', () => { + const ctx = createMockContext('/mock/extension'); + const logger = createMockLogger(); + const registrar = new AgentRegistrar(ctx, logger); + const status = registrar.getStatus(); + + expect(status).toHaveProperty('enabled'); + expect(status).toHaveProperty('bundledDir'); + expect(status).toHaveProperty('additionalDirs'); + expect(status).toHaveProperty('effectiveLocations'); + expect(status.bundledDir).toBe('/mock/extension/agents'); + }); +}); diff --git a/extensions/vscode/test/customizations/bundle-customizations.test.ts b/extensions/vscode/test/customizations/bundle-customizations.test.ts new file mode 100644 index 00000000..0e96eb7e --- /dev/null +++ b/extensions/vscode/test/customizations/bundle-customizations.test.ts @@ -0,0 +1,170 @@ +/** + * Tests for bundle-customizations.js + * + * Runs the bundler's exported `runBundle()` function in an isolated temp + * directory structure to verify default copying, manifest generation, + * overlay support, and graceful handling of absent whitelisted files. + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { + existsSync, + mkdirSync, + mkdtempSync, + readFileSync, + rmSync, + writeFileSync, +} from 'fs'; +import { join, resolve } from 'path'; +import { fileURLToPath } from 'url'; + +// Resolve the actual script location +const __repoRoot = resolve(fileURLToPath(import.meta.url), '..', '..', '..', '..', '..'); + +// Import the bundler dynamically (ES module) +async function importBundler() { + const bundlerPath = resolve(__repoRoot, 'extensions', 'vscode', 'scripts', 'bundle-customizations.js'); + return import(bundlerPath) as Promise<{ runBundle: (opts: { extensionRoot: string; customizationsDir?: string }) => Promise<{ agents: string[]; prompts: string[]; skills: string[] }> }>; +} + +describe('bundle-customizations', () => { + let tmp: string; + + beforeEach(() => { + // Use project-local .tmp/ rather than process.cwd() to avoid polluting + // the repo root and to match the convention used elsewhere in the + // monorepo. .tmp/ is gitignored at the repo root. + const tmpRoot = resolve(__repoRoot, '.tmp'); + mkdirSync(tmpRoot, { recursive: true }); + tmp = mkdtempSync(join(tmpRoot, 'bundle-test-')); + }); + + afterEach(() => { + if (existsSync(tmp)) rmSync(tmp, { recursive: true, force: true }); + }); + + it('copies agent files to agents/ output dir', async () => { + const fakeDeep = join(tmp, 'fake-repo', 'extensions', 'vscode'); + mkdirSync(fakeDeep, { recursive: true }); + // Copy customizations into fakeDeep + const customizationsDir = join(fakeDeep, 'customizations'); + const agentsDir = join(customizationsDir, 'agents'); + mkdirSync(agentsDir, { recursive: true }); + writeFileSync( + join(agentsDir, 'codeql-query-developer.agent.md'), + '---\nname: codeql-query-developer\n---\n', + ); + // Config that references real repo paths + writeFileSync( + join(customizationsDir, 'bundle-customizations.config.js'), + `export const prompts = ['ql-tdd-basic.prompt.md'];\nexport const skills = ['validate-ql-mcp-server-tools-queries', 'nonexistent-skill'];\n`, + ); + + // Create the server/src/prompts dir + const serverPromptsDir = join(tmp, 'fake-repo', 'server', 'src', 'prompts'); + mkdirSync(serverPromptsDir, { recursive: true }); + // Copy a real prompt + const realPrompt = join(__repoRoot, 'server', 'src', 'prompts', 'ql-tdd-basic.prompt.md'); + if (existsSync(realPrompt)) { + const content = readFileSync(realPrompt, 'utf8'); + writeFileSync(join(serverPromptsDir, 'ql-tdd-basic.prompt.md'), content); + } + + // Create skills dir + const skillsDir = join(tmp, 'fake-repo', '.github', 'skills', 'validate-ql-mcp-server-tools-queries'); + mkdirSync(skillsDir, { recursive: true }); + const realSkill = join(__repoRoot, '.github', 'skills', 'validate-ql-mcp-server-tools-queries', 'SKILL.md'); + if (existsSync(realSkill)) { + writeFileSync(join(skillsDir, 'SKILL.md'), readFileSync(realSkill, 'utf8')); + } + + const { runBundle } = await importBundler(); + const manifest = await runBundle({ extensionRoot: fakeDeep }); + + // Agents dir should exist + expect(existsSync(join(fakeDeep, 'agents', 'codeql-query-developer.agent.md'))).toBe(true); + + // Prompt should be present + expect(existsSync(join(fakeDeep, 'prompts', 'ql-tdd-basic.prompt.md'))).toBe(true); + + // Skill should be present + expect(existsSync(join(fakeDeep, 'skills', 'validate-ql-mcp-server-tools-queries', 'SKILL.md'))).toBe(true); + + // Manifest emitted + expect(existsSync(join(fakeDeep, 'dist-customizations-manifest.json'))).toBe(true); + expect(manifest.agents).toContain('agents/codeql-query-developer.agent.md'); + expect(manifest.prompts).toContain('prompts/ql-tdd-basic.prompt.md'); + expect(manifest.skills).toContain('skills/validate-ql-mcp-server-tools-queries/SKILL.md'); + }); + + it('warns but does not fail when whitelisted files are absent', async () => { + const fakeDeep = join(tmp, 'fake-repo2', 'extensions', 'vscode'); + mkdirSync(join(fakeDeep, 'customizations', 'agents'), { recursive: true }); + writeFileSync( + join(fakeDeep, 'customizations', 'bundle-customizations.config.js'), + `export const prompts = ['nonexistent.prompt.md'];\nexport const skills = ['nonexistent-skill'];\n`, + ); + + const { runBundle } = await importBundler(); + + // Should not throw + await expect(runBundle({ extensionRoot: fakeDeep })).resolves.toBeDefined(); + + const manifestPath = join(fakeDeep, 'dist-customizations-manifest.json'); + expect(existsSync(manifestPath)).toBe(true); + const manifest = JSON.parse(readFileSync(manifestPath, 'utf8')) as { agents: string[]; prompts: string[]; skills: string[] }; + // Nothing should have been copied for prompts/skills + expect(manifest.prompts).toEqual([]); + expect(manifest.skills).toEqual([]); + }); + + it('overlay: replaces existing file with warning and adds net-new files', async () => { + const fakeDeep = join(tmp, 'fake-repo3', 'extensions', 'vscode'); + mkdirSync(join(fakeDeep, 'customizations', 'agents'), { recursive: true }); + writeFileSync( + join(fakeDeep, 'customizations', 'agents', 'codeql-query-developer.agent.md'), + '---\nname: codeql-query-developer\n---\n# Default\n', + ); + writeFileSync( + join(fakeDeep, 'customizations', 'bundle-customizations.config.js'), + `export const prompts = [];\nexport const skills = [];\n`, + ); + + // Create an overlay directory + const overlayDir = join(tmp, 'overlay'); + mkdirSync(join(overlayDir, 'agents'), { recursive: true }); + // Override the existing agent + writeFileSync( + join(overlayDir, 'agents', 'codeql-query-developer.agent.md'), + '---\nname: codeql-query-developer\n---\n# Override\n', + ); + // Add a net-new agent + writeFileSync( + join(overlayDir, 'agents', 'team-agent.agent.md'), + '---\nname: team-agent\n---\n# Team\n', + ); + + const { runBundle } = await importBundler(); + + // Capture console.warn calls + const warnings: string[] = []; + const originalWarn = console.warn; + console.warn = (...args: unknown[]) => { warnings.push(String(args[0])); }; + + try { + await runBundle({ extensionRoot: fakeDeep, customizationsDir: overlayDir }); + + // Replacement should have warned + expect(warnings.some((w) => w.includes('Overlay replaces bundled file'))).toBe(true); + + // Override file has new content + const overrideContent = readFileSync(join(fakeDeep, 'agents', 'codeql-query-developer.agent.md'), 'utf8'); + expect(overrideContent).toContain('# Override'); + + // Net-new file should exist + expect(existsSync(join(fakeDeep, 'agents', 'team-agent.agent.md'))).toBe(true); + } finally { + console.warn = originalWarn; + } + }); +}); diff --git a/extensions/vscode/test/suite/agents.integration.test.ts b/extensions/vscode/test/suite/agents.integration.test.ts new file mode 100644 index 00000000..fe6aba87 --- /dev/null +++ b/extensions/vscode/test/suite/agents.integration.test.ts @@ -0,0 +1,134 @@ +/** + * Integration tests for built-in custom agents. + * + * These run inside the Extension Development Host with the REAL VS Code API. + * They verify the agents/ directory is bundled, the .agent.md files exist, + * and that chat.agentFilesLocations is updated correctly. + */ + +import * as assert from 'assert'; +import * as fs from 'fs'; +import * as path from 'path'; +import * as vscode from 'vscode'; + +const EXTENSION_ID = 'advanced-security.vscode-codeql-development-mcp-server'; + +suite('Agents Integration Tests', () => { + let ext: vscode.Extension; + + suiteSetup(async () => { + const found = vscode.extensions.getExtension(EXTENSION_ID); + assert.ok(found, `Extension ${EXTENSION_ID} not found`); + ext = found; + if (!ext.isActive) { + await ext.activate(); + } + }); + + test('Extension agents/ directory exists', () => { + const agentsDir = path.join(ext.extensionPath, 'agents'); + assert.ok(fs.existsSync(agentsDir), `agents/ dir should exist at ${agentsDir}`); + }); + + test('codeql-query-developer.agent.md exists and has correct name frontmatter', () => { + const agentPath = path.join(ext.extensionPath, 'agents', 'codeql-query-developer.agent.md'); + assert.ok(fs.existsSync(agentPath), `${agentPath} should exist`); + const content = fs.readFileSync(agentPath, 'utf8'); + assert.ok(content.includes('name: codeql-query-developer'), 'Should contain name frontmatter'); + assert.ok(!content.includes('model:'), 'Should NOT contain model: key'); + }); + + test('codeql-workshop-author.agent.md exists and has correct name frontmatter', () => { + const agentPath = path.join(ext.extensionPath, 'agents', 'codeql-workshop-author.agent.md'); + assert.ok(fs.existsSync(agentPath), `${agentPath} should exist`); + const content = fs.readFileSync(agentPath, 'utf8'); + assert.ok(content.includes('name: codeql-workshop-author'), 'Should contain name frontmatter'); + assert.ok(!content.includes('model:'), 'Should NOT contain model: key'); + }); + + test('dist-customizations-manifest.json exists and lists expected files', () => { + const manifestPath = path.join(ext.extensionPath, 'dist-customizations-manifest.json'); + assert.ok(fs.existsSync(manifestPath), 'dist-customizations-manifest.json should exist'); + const manifest = JSON.parse(fs.readFileSync(manifestPath, 'utf8')); + assert.ok(Array.isArray(manifest.agents), 'manifest.agents should be an array'); + assert.ok(Array.isArray(manifest.prompts), 'manifest.prompts should be an array'); + assert.ok(Array.isArray(manifest.skills), 'manifest.skills should be an array'); + assert.ok( + manifest.agents.some((a: string) => a.includes('codeql-query-developer')), + 'Manifest should list codeql-query-developer agent', + ); + }); + + test('codeql-mcp.showAgentsStatus command resolves without throwing', async () => { + await assert.doesNotReject( + vscode.commands.executeCommand('codeql-mcp.showAgentsStatus'), + 'showAgentsStatus command should not throw', + ); + }); + + test('Toggling codeql-mcp.agents.enabled = false removes bundled dir; re-enabling restores it', async () => { + const agentsDir = path.join(ext.extensionPath, 'agents'); + const cfg = vscode.workspace.getConfiguration('codeql-mcp'); + const chatCfg = vscode.workspace.getConfiguration('chat'); + + // Save original values + const originalEnabled = cfg.get('agents.enabled', true); + const originalLocations = chatCfg.get>('agentFilesLocations', {}); + + try { + // Disable agents + await cfg.update('agents.enabled', false, vscode.ConfigurationTarget.Global); + // Give the registrar time to react + await new Promise((resolve) => setTimeout(resolve, 200)); + + const locationsAfterDisable = vscode.workspace.getConfiguration('chat') + .get>('agentFilesLocations', {}); + const containsBundledAfterDisable = Object.keys(locationsAfterDisable).some( + (k) => path.normalize(k) === path.normalize(agentsDir), + ); + assert.strictEqual(containsBundledAfterDisable, false, 'Bundled dir should be removed when disabled'); + + // Re-enable + await cfg.update('agents.enabled', true, vscode.ConfigurationTarget.Global); + await new Promise((resolve) => setTimeout(resolve, 200)); + + const locationsAfterEnable = vscode.workspace.getConfiguration('chat') + .get>('agentFilesLocations', {}); + const containsBundledAfterEnable = Object.keys(locationsAfterEnable).some( + (k) => path.normalize(k) === path.normalize(agentsDir), + ); + assert.strictEqual(containsBundledAfterEnable, true, 'Bundled dir should be restored when re-enabled'); + } finally { + // Restore original config + await cfg.update('agents.enabled', originalEnabled, vscode.ConfigurationTarget.Global); + await chatCfg.update('agentFilesLocations', originalLocations, vscode.ConfigurationTarget.Global); + } + }); + + test('Setting codeql-mcp.additionalAgentDirs appends the dir', async () => { + const tmpRoot = path.resolve(ext.extensionPath, '..', '..', '..', '.tmp'); + fs.mkdirSync(tmpRoot, { recursive: true }); + const tmpDir = fs.mkdtempSync(path.join(tmpRoot, 'codeql-mcp-test-')); + const cfg = vscode.workspace.getConfiguration('codeql-mcp'); + const chatCfg = vscode.workspace.getConfiguration('chat'); + + const originalAdditional = cfg.get('additionalAgentDirs', []); + const originalLocations = chatCfg.get>('agentFilesLocations', {}); + + try { + await cfg.update('additionalAgentDirs', [tmpDir], vscode.ConfigurationTarget.Global); + await new Promise((resolve) => setTimeout(resolve, 200)); + + const locations = vscode.workspace.getConfiguration('chat') + .get>('agentFilesLocations', {}); + const containsTmpDir = Object.keys(locations).some( + (k) => path.normalize(k) === path.normalize(tmpDir), + ); + assert.ok(containsTmpDir, `tmpDir ${tmpDir} should be in agentFilesLocations`); + } finally { + await cfg.update('additionalAgentDirs', originalAdditional, vscode.ConfigurationTarget.Global); + await chatCfg.update('agentFilesLocations', originalLocations, vscode.ConfigurationTarget.Global); + try { fs.rmdirSync(tmpDir); } catch { /* ignore */ } + } + }); +});