Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,15 @@ release cadence.

_Changes on `main` since the latest tagged release that have not yet been included in a stable release._

### Added

#### VS Code Extension

- **Built-in portable custom agents** — The extension now ships two `.agent.md` custom agents (`codeql-query-developer`, `codeql-workshop-author`) bundled inside the VSIX. On activation, the extension registers its `agents/` directory in `chat.agentFilesLocations` so both agents are immediately discoverable in VS Code Copilot Chat without any manual configuration. No specific model is required — users choose their own.
- **User-extensibility hooks** — Two new settings allow teams and individuals to extend the bundled agents at runtime (`codeql-mcp.additionalAgentDirs: string[]`) or disable the built-in registration entirely (`codeql-mcp.agents.enabled: boolean`, default `true`). A `--customizations-dir` CLI flag (or `CODEQL_MCP_CUSTOMIZATIONS_DIR` env var) on `bundle:customizations` enables building custom VSIXes with overlay agents, prompts, and skills.
- **`codeql-mcp.showAgentsStatus` command** — New Command Palette entry (**CodeQL MCP: Show Built-in Custom Agents Status**) that reports the enabled state, bundled directory, additional directories, and effective `chat.agentFilesLocations` entries.
- **Bundled prompts and skills** — Four MCP server prompts (`ql-tdd-basic`, `ql-tdd-advanced`, `tools-query-workflow`, `workshop-creation-workflow`) and two skills (`create-codeql-query-development-workshop`, `validate-ql-mcp-server-tools-queries`) are now copied into the VSIX as static contribution points (`chatPromptFiles`, `chatSkills`) so they are available to Copilot Chat without the MCP server running.

## [v2.25.4] — 2026-05-08

### Highlights
Expand Down
6 changes: 6 additions & 0 deletions extensions/vscode/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,11 @@ server/
coverage/
*.vsix

# Customizations bundle output (generated by scripts/bundle-customizations.js)
agents/
prompts/
skills/
dist-customizations-manifest.json

# @vscode/test-cli downloads and runtime data
.vscode-test/
4 changes: 4 additions & 0 deletions extensions/vscode/.vscodeignore
Original file line number Diff line number Diff line change
Expand Up @@ -27,3 +27,7 @@ coverage/**
# Bundled server: exclude test/examples content
server/ql/*/tools/test/**
server/ql/*/examples/**

# Customization sources (bundled outputs in agents/, prompts/, skills/ ship in VSIX)
customizations/**
examples/**
84 changes: 69 additions & 15 deletions extensions/vscode/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,13 +33,65 @@ On activation (`onStartupFinished`), the extension:
1. **Auto-installs** the `codeql-development-mcp-server` npm package (unless `codeql-mcp.autoInstall` is `false`).
2. **Registers an MCP server definition** (`ql-mcp`) so VS Code's Copilot/MCP integration can discover and launch it.
3. **Watches** the CodeQL extension's storage paths for databases, query results, and MRVA results, passing them to the MCP server as environment variables.
4. **Registers built-in custom agents** (`codeql-query-developer`, `codeql-workshop-author`) in `chat.agentFilesLocations` so they are discoverable in VS Code Copilot Chat.

## Built-in Custom Agents

The extension ships two portable `.agent.md` custom agents that appear in VS Code's Copilot Chat agent picker:

| Agent | Description |
| ----- | ----------- |
| `codeql-query-developer` | Develop CodeQL queries, libraries, and tests using TDD with the `ql-mcp` MCP server tools. |
| `codeql-workshop-author` | Create CodeQL query development workshops from production-grade queries. |

Both agents use the bundled MCP server tools (`ql-mcp/*`), prompts, and skills that ship with the extension. **No specific model is required** — you choose your own model in VS Code Copilot Chat.

### Enabling / Disabling the Built-in Agents

Agents are enabled by default. To disable them, set:

```json
"codeql-mcp.agents.enabled": false
```

This removes the bundled `agents/` directory from `chat.agentFilesLocations` so the agents are no longer discoverable in Copilot Chat.

### Extending at Runtime — Additional Agent Directories

To add team or personal `.agent.md` files without rebuilding the extension, use:

```json
"codeql-mcp.additionalAgentDirs": ["/path/to/your/team-agents"]
```

This appends the directory to `chat.agentFilesLocations` alongside the bundled agents.

### Extending at Build Time — Custom VSIX

To bundle your own agents, prompts, and skills into a custom VSIX:

```bash
cd extensions/vscode
npm run bundle:customizations -- --customizations-dir=./examples/team-customizations
npm run package
```

Or via environment variable:

```bash
CODEQL_MCP_CUSTOMIZATIONS_DIR=./examples/team-customizations npm run bundle:customizations
```

See [`examples/team-customizations/`](./examples/team-customizations/README.md) for a complete overlay example.

## Configuration

All settings are under the `codeql-mcp` namespace in VS Code settings:

| Setting | Default | Description |
| ------------------------------------------ | ---------- | ------------------------------------------------------------------- |
| `codeql-mcp.agents.enabled` | `true` | Register the bundled custom agents in `chat.agentFilesLocations`. |
| `codeql-mcp.additionalAgentDirs` | `[]` | Additional `.agent.md` directories appended to `chat.agentFilesLocations`. |
| `codeql-mcp.autoInstall` | `true` | Auto-install/update the MCP server on activation. |
| `codeql-mcp.serverVersion` | `"latest"` | npm version to install (`"latest"` for most recent). |
| `codeql-mcp.serverCommand` | `"node"` | Command to launch the server. Override to `"npx"` or a custom path. |
Expand All @@ -55,26 +107,28 @@ All settings are under the `codeql-mcp` namespace in VS Code settings:

Available from the Command Palette (`Cmd+Shift+P` / `Ctrl+Shift+P`):

| Command | Description |
| ------------------------------------------------- | ----------------------------------------------- |
| **CodeQL MCP: Reinstall MCP Server** | Re-download and install the server package. |
| **CodeQL MCP: Reinstall CodeQL Tool Query Packs** | Re-install the bundled CodeQL tool query packs. |
| **CodeQL MCP: Show Status** | Display current server status. |
| **CodeQL MCP: Show Logs** | Open the server log output. |
| Command | Description |
| -------------------------------------------------------- | ----------------------------------------------- |
| **CodeQL MCP: Reinstall MCP Server** | Re-download and install the server package. |
| **CodeQL MCP: Reinstall CodeQL Tool Query Packs** | Re-install the bundled CodeQL tool query packs. |
| **CodeQL MCP: Show Built-in Custom Agents Status** | Show which agent dirs are registered. |
| **CodeQL MCP: Show Status** | Display current server status. |
| **CodeQL MCP: Show Logs** | Open the server log output. |

## Development

### npm Scripts

| Script | What it does | When to use |
| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------- |
| `npm run package` | **Builds everything and produces the `.vsix`**. Internally runs `vscode:prepublish` (clean → lint → bundle → bundle:server) then `vsce package`. | **Building a distributable `.vsix`.** |
| `npm run build` | `clean` → `lint` → `bundle` (extension only, no server). | Development builds without packaging. |
| `npm run bundle` | esbuild the extension (no lint, no clean). | Fast iteration during development. |
| `npm run watch` | Rebuild the extension on file changes. | Active development. |
| `npm run test` | Run unit tests with Vitest. | Validating changes. |
| `npm run test:coverage` | Run unit tests with coverage. | CI / pre-merge validation. |
| `npm run lint` | Run ESLint on `src/` and `test/`. | Checking code style. |
| Script | What it does | When to use |
| ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------- |
| `npm run package` | **Builds everything and produces the `.vsix`**. Internally runs `vscode:prepublish` (clean → lint → bundle → bundle:server → bundle:customizations) then `vsce package`. | **Building a distributable `.vsix`.** |
| `npm run build` | `clean` → `lint` → `bundle` (extension only, no server or customizations). | Development builds without packaging. |
| `npm run bundle` | esbuild the extension (no lint, no clean). | Fast iteration during development. |
| `npm run bundle:customizations`| Copy bundled agents/prompts/skills to output dirs and write the manifest. | After modifying agent/prompt/skill sources. |
| `npm run watch` | Rebuild the extension on file changes. | Active development. |
| `npm run test` | Run unit tests with Vitest. | Validating changes. |
| `npm run test:coverage` | Run unit tests with coverage. | CI / pre-merge validation. |
| `npm run lint` | Run ESLint on `src/` and `test/`. | Checking code style. |

> **Note:** `vscode:prepublish` is a lifecycle hook invoked automatically by `vsce package` — you should not need to run it directly.

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
---
name: codeql-query-developer
description: "Develop CodeQL queries, libraries, and tests with TDD via the ql-mcp server."
tools: ['ql-mcp/*', 'edit', 'read', 'search', 'todo']
---

# `codeql-query-developer` Agent

Develops, tests, and validates CodeQL queries, libraries, and tests using the QL MCP Server tools.

## Core Capabilities

- Uses `ql-mcp/*` tools to create and manage CodeQL databases from source code.
- Follows test-driven development (TDD): writes tests with expected results first, then implements queries to pass them.
- Uses `ql-mcp/*` tools to run queries against databases, execute query unit tests, and generate query logs for debugging.
- Organizes queries, libraries, and tests following CodeQL pack conventions (`qlpack.yml`, `codeql-workspace.yml`).
- Documents query purpose, logic, and usage with clear QL comments.
- ALWAYS uses verbose help (`codeql <subcommand> -h -vv`) when learning about `codeql` CLI commands.
- NEVER makes anything up about CodeQL semantics or database schema.
- NEVER assumes query behavior without testing against actual databases.

## TDD Workflow

1. **Understand the goal** — clarify what the query should detect and for which language.
2. **Create test code** — write test source files that contain positive and negative examples.
3. **Extract a test database** — use `ql-mcp/codeql_create_database` or `ql-mcp/codeql_query_run` to build a DB.
4. **Write `.qlref` / `.expected` test files** — specify expected results before writing query logic.
5. **Implement the query** — write the `.ql` file to make the tests pass.
6. **Run tests** — use `ql-mcp/codeql_test_run` to execute the unit tests; iterate until 100% pass.
7. **Validate** — run the query against real databases; inspect results; refine as needed.
8. **Document** — add `@name`, `@description`, `@kind`, `@id`, `@tags` metadata to the query.

## MCP Tool Usage

Use the bundled `ql-mcp/*` tools for all CodeQL operations:

- `ql-mcp/codeql_create_database` — create a CodeQL database from source.
- `ql-mcp/codeql_query_run` — run a query against a database.
- `ql-mcp/codeql_test_run` — run CodeQL unit tests.
- `ql-mcp/codeql_query_explain` — explain a query's structure.
- `ql-mcp/find_codeql_query_files` — locate query files in the workspace.
- `ql-mcp/codeql_pack_install` — install QL pack dependencies.

## Bundled Skills and Prompts

The following bundled resources are available in the extension and provide detailed step-by-step workflows:

- **Skill `create-codeql-query-development-workshop`** — reference for structured query development.
- **Skill `validate-ql-mcp-server-tools-queries`** — validate PrintAST, PrintCFG, and CallGraph tools.
- **Prompt `ql-tdd-basic`** — basic TDD workflow for simple CodeQL queries.
- **Prompt `ql-tdd-advanced`** — advanced TDD patterns for data-flow and taint-tracking queries.
- **Prompt `tools-query-workflow`** — workflow for using MCP tool queries (PrintAST, PrintCFG, CallGraph).

## Quality Standards

- All solution queries must compile without errors.
- All unit tests must pass at 100%.
- Expected results must be accurate (verified against real test databases).
- Queries must include complete `@name`, `@description`, `@kind`, `@id`, `@tags` metadata.
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
---
name: codeql-workshop-author
description: "Create CodeQL query development workshops from production-grade queries."
tools: ['ql-mcp/*', 'edit', 'read', 'search', 'todo']
handoffs:
- agent: codeql-query-developer
label: Develop and Test Query
prompt: 'Develop and test a CodeQL query using TDD methodology. Follow the `ql-tdd-basic` or `ql-tdd-advanced` prompt workflow and return the validated query file path and test results when complete.'
send: false
---

# `codeql-workshop-author` Agent

Creates comprehensive CodeQL query development workshops from production-grade queries using the QL MCP Server tools.

## Core Capabilities

- Uses `ql-mcp/*` tools to analyze production queries and create workshop materials.
- Follows the bundled `create-codeql-query-development-workshop` skill to generate workshops from production queries.
- Validates AST/CFG tools using the bundled `validate-ql-mcp-server-tools-queries` skill.
- Decomposes queries into 4–8 logical learning stages that guide learners from simple to complex.
- Generates exercise queries, solution queries, unit tests, AST/CFG visualizations, and README documentation.
- ALWAYS uses verbose help (`codeql <subcommand> -h -vv`) when learning about `codeql` CLI commands.
- NEVER makes anything up about CodeQL semantics or database schema.
- NEVER assumes query behavior without testing against actual databases.

## Workshop Generation Process

1. **Analyze source query** — use `ql-mcp/find_codeql_query_files` and `ql-mcp/codeql_query_explain` to understand the production query.
2. **Prepare environment** — run `codeql pack install` on solutions and solutions-tests directories; run any `initialize-qltests.sh` scripts.
3. **Validate AST/CFG tools** — use the bundled `validate-ql-mcp-server-tools-queries` skill to confirm PrintAST, PrintCFG, and CallGraph return non-empty output. **Fail if any query returns empty results.**
4. **Plan stages** — decompose the query into 4–8 logical learning stages.
5. **Create workshop structure** — set up directories, `qlpack.yml` files, and `codeql-workspace.yml`.
6. **Generate solution stages** — for each stage, delegate to `codeql-query-developer` (via the **Develop and Test Query** handoff) to create and validate the solution query.
7. **Create exercise queries** — remove implementation details from solutions; add scaffolding, `// TODO` hints, and `select` stubs.
8. **Generate enrichments** — create AST/CFG graphs (from tool output), build scripts, and documentation.
9. **Final validation** — run all solution tests; confirm 100% pass rate before declaring the workshop complete.

## Workshop Structure

```
<workshop-name>/
exercises/ # Student exercise queries (incomplete, with scaffolding)
exercises-tests/ # Unit tests for exercises
solutions/ # Complete solution queries
solutions-tests/ # Unit tests for solutions (must pass 100%)
tests-common/ # Shared test code and databases
graphs/ # AST/CFG visualizations
README.md # Workshop guide
build-databases.sh # Database creation script
codeql-workspace.yml
```

## Decomposition Strategies

- **Syntactic → Semantic** — Start with syntax, add type, control flow, then data flow.
- **Local → Global** — Start local, expand to cross-procedural analysis.
- **Simple → Filtered** — High recall first, then refine with filters.
- **Building Blocks** — Define helpers, combine into sources/sinks, connect with flow.

## Bundled Skills and Prompts

- **Skill `create-codeql-query-development-workshop`** — full step-by-step workshop creation workflow.
- **Skill `validate-ql-mcp-server-tools-queries`** — AST/CFG/CallGraph validation protocol.
- **Prompt `workshop-creation-workflow`** — structured prompt for workshop generation from a production query.
- **Prompt `ql-tdd-advanced`** — advanced TDD patterns for data-flow and taint-tracking queries.

## Quality Standards

- All solution queries compile without errors.
- All solution tests pass at 100%.
- Exercise queries have appropriate scaffolding (not empty, not complete).
- Expected results progress logically from stage to stage.
- Test code covers positive, negative, and edge cases.
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
/**
* bundle-customizations.config.js
*
* Whitelist of prompts and skills to bundle into the VS Code extension.
* Prompts are sourced from server/src/prompts/.
* Skills are sourced from .github/skills/<name>/SKILL.md.
*
* Missing entries are silently skipped with a console.warn — the build
* never fails due to absent optional files.
*/

export const prompts = [
'ql-tdd-basic.prompt.md',
'ql-tdd-advanced.prompt.md',
'tools-query-workflow.prompt.md',
'workshop-creation-workflow.prompt.md',
];

export const skills = [
'create-codeql-query-development-workshop',
'create-codeql-query-tdd-generic',
'validate-ql-mcp-server-tools-queries',
];
Loading