VS Code extension: ship portable, built-in custom agents (codeql-query-developer, codeql-workshop-author) with user-extensible packaging

[data-douser]

Summary

Bundle two query-author-facing custom agents as part of the extensions/vscode/ VSIX so users get a turnkey, named-agent experience the moment the extension is installed — no repo .github/agents/*.md files required. Mirror the user-provided-dir paradigm from #2 (--prompts-dir) so teams can override or extend with their own .agent.md / .prompt.md / SKILL.md files at both packaging time and runtime.

This complements (does not replace) the existing repo-local .github/agents/*.md files used by maintainers of this MCP server.

Why

Repo-local .github/agents/*.md files are not portable — they only exist for users working inside the codeql-development-mcp-server checkout. End users who install the VSIX into their own CodeQL workspace currently see the ql-mcp/* MCP tools but get no curated agent personas, slash-command prompts, or skills out of the box. Shipping these as VSIX content closes that gap and aligns with VS Code's documented chatPromptFiles / chatSkills contribution points and the chat.agentFilesLocations mechanism for .agent.md files.

Implementation methodology: TDD (mandatory)

Implementation must follow test-driven development. The highest-value tests for this work are extension-level integration tests that exercise the activated VSIX inside a real VS Code instance — write these first, watch them fail, then implement until they pass.

• Primary suite — @vscode/test-electron integration tests in extensions/vscode/test/integration/:
  • Activate the extension in a temp workspace and assert:
    • The bundled agents/ directory is registered with VS Code (present in the effective chat.agentFilesLocations value).
    • The two shipped .agent.md files are discoverable from context.extensionPath.
    • Contributed prompt files and skills (per package.json chatPromptFiles / chatSkills) are visible to chat APIs (best-effort via vscode.lm / file probes).
    • Toggling codeql-mcp.agents.enabled = false removes the registered directory; re-enabling restores it.
    • Adding a path to codeql-mcp.additionalAgentDirs appends it to chat.agentFilesLocations.
    • Deactivation cleans up registrations idempotently (no stale entries on re-activate).
    • The new codeql-mcp.showAgentsStatus command runs and reports bundled + extra paths.
  • These are the tests we trust most. They must be the primary acceptance gate.
• Secondary — Vitest unit tests in extensions/vscode/test/customizations/:
  • bundle-customizations.test.ts — overlay semantics, collision warnings, manifest output.
  • agent-registrar.test.ts — mocked vscode.workspace.getConfiguration, ConfigurationTarget choice, cross-platform path normalization, dedupe, idempotency.
• Tertiary — packaging smoke tests: npm run package succeeds; resulting .vsix contains agents/, prompts/, skills/ but not the customizations/ source dir.

Workflow per phase: red (integration test asserting the missing behavior) → green (minimal implementation) → refactor. Do not advance to the next phase until the integration tests for the current phase pass.

Model choice — user-controlled, never forced

Neither shipped .agent.md file may set a model: frontmatter key. Agent behavior must respect whichever model the user selects in the chat model picker (Claude Sonnet, GPT, etc.). Likewise, no code path may default to or coerce a specific model. This is a non-negotiable design requirement: do not copy the model: Claude Opus 4.6 (...) line that appears in the repo-local .github/agents/*.md files.

If a model recommendation is helpful, surface it only in human-readable docs (README), never in agent frontmatter or runtime configuration.

Scope decisions (confirmed)

• Ship two agents only (renamed, codeql-* prefix):
  • codeql-query-developer (derived from mcp-enabled-ql-query-developer)
  • codeql-workshop-author (derived from mcp-enabled-ql-workshop-developer)
• Out of scope to ship (stay repo-internal in .github/agents/): ql-mcp-tool-developer, ql-mcp-tool-tester, ql-agent-skills-developer.
• All five existing .github/agents/*.md files stay untouched.
• Prompts + skills are in scope: contribute via documented contributes.chatPromptFiles and contributes.chatSkills.
• Hybrid registration for agents: bundle in VSIX + programmatically add to chat.agentFilesLocations (no documented chatAgents contribution point exists as of 2026-05; try one speculatively, fall back to programmatic).
• User extensibility: build-time overlay (npm run package -- --customizations-dir=…) and runtime setting codeql-mcp.additionalAgentDirs.

Architecture (3 layers)

1. Build-time bundling

New script extensions/vscode/scripts/bundle-customizations.js:

• Copies "source of truth" agent files from a new tracked dir extensions/vscode/customizations/agents/ → packaged agents/.
• Copies a whitelisted subset of ../../server/src/prompts/*.prompt.md → packaged prompts/.
• Copies a whitelisted subset of ../../.github/skills/*/SKILL.md (starting with create-codeql-query-tdd-generic, create-codeql-query-development-workshop, validate-ql-mcp-server-tools-queries) → packaged skills/<name>/SKILL.md.
• Optional overlay: --customizations-dir <path> (or env CODEQL_MCP_CUSTOMIZATIONS_DIR) copies <path>/{agents,prompts,skills} over the bundled files (collision → warn + replace).
• Emits dist-customizations-manifest.json listing all bundled files (consumed by package.json contribution generation + runtime checks + tests).
• Whitelist config lives at customizations/bundle-customizations.config.js.
• Wired into vscode:prepublish after bundle:server.
• .vscodeignore excludes the customizations/ source dir; only generated agents/, prompts/, skills/ ship in the VSIX.
• clean script removes the generated output dirs.

2. package.json contribution points

• Add contributes.chatPromptFiles[] — one entry per packaged prompt file (auto-generated from the manifest).
• Add contributes.chatSkills[] — one entry per packaged SKILL.md.
• Speculatively try contributes.chatAgents[] if/when supported; remove if vsce package validation rejects.
• Add new settings under codeql-mcp:
  • codeql-mcp.agents.enabled (boolean, default true)
  • codeql-mcp.additionalAgentDirs (string[], default [])
• Add new command codeql-mcp.showAgentsStatus.

3. Runtime agent registration

New extensions/vscode/src/customizations/agent-registrar.ts:

• On activation, computes the bundled agents/ absolute path from context.extensionUri.
• Reads current chat.agentFilesLocations, appends (1) the bundled dir and (2) any additionalAgentDirs, using ConfigurationTarget.Workspace only when a workspace is open, otherwise Global. Path comparison is cross-platform normalized before adding.
• Idempotent; removes its own entries on deactivation.
• Re-fires on onDidChangeConfiguration('codeql-mcp.agents') and onDidChangeWorkspaceFolders.
• Defensive: try/catch with warn-level log; never blocks activation.
• Wired into src/extension.ts activate() alongside the existing McpProvider registration; disposed via the existing disposables[] pattern.

Shipped agent definitions

Both .agent.md files use VS Code custom-agent frontmatter. Slimmed from the existing repo versions for portability:

• customizations/agents/codeql-query-developer.agent.md
  • description: "Develop CodeQL queries, libraries, and tests with TDD via the ql-mcp server."
  • tools: ['ql-mcp/*', 'edit', 'read', 'search', 'todo']
  • No model: key (user-controlled).
  • No handoffs in v1 (handoff targets only resolve if both agents are enabled).
  • Body: TDD workflow, MCP tool usage, references bundled skills and prompts by name.
• customizations/agents/codeql-workshop-author.agent.md
  • description: "Create CodeQL query development workshops from production-grade queries."
  • tools: ['ql-mcp/*', 'edit', 'read', 'search', 'todo']
  • No model: key (user-controlled).
  • One-way handoff → codeql-query-developer (label "Develop and Test Query"). No reverse handoff (avoid loops).
  • Body: workshop generation workflow; references bundled skills.

Phased plan (TDD throughout)

Each phase begins by writing failing integration tests, then implementation, then refactor.

Phase A — Bundling pipeline (foundation)

1. Write failing integration/unit tests asserting the bundled VSIX layout (agents/prompts/skills present; source customizations/ absent).
2. Add tracked extensions/vscode/customizations/ with the two .agent.md files (no model: key) and bundle-customizations.config.js whitelist.
3. Implement scripts/bundle-customizations.js with overlay support + manifest output.
4. Wire bundle:customizations into vscode:prepublish; update .vscodeignore and clean.

Phase B — Contribution points (declarative wiring) — depends on A

1. Write failing tests asserting that contributed prompt files and skills are discoverable post-activation.
2. Add chatPromptFiles + chatSkills arrays to package.json (generated from the manifest).
3. Add new settings + command to contributes.configuration / contributes.commands.
4. Try contributes.chatAgents speculatively; remove if rejected by vsce.

Phase C — Runtime agent registration — parallel with B

1. Write failing integration tests for chat.agentFilesLocations registration, toggle behavior, additionalAgentDirs, idempotent deactivation, and showAgentsStatus.
2. Implement src/customizations/agent-registrar.ts.
3. Wire into src/extension.ts; register the codeql-mcp.showAgentsStatus command.

Phase D — User-overlay packaging story — depends on A, B, C

1. Write failing tests for the --customizations-dir overlay path (collision warnings, override precedence).
2. Document the flow in extensions/vscode/README.md.
3. Add extensions/vscode/examples/team-customizations/ showing override + add-new patterns.
4. Document the runtime alternative (codeql-mcp.additionalAgentDirs).

Phase E — Docs, changelog (final polish)

1. Update extensions/vscode/README.md: "Built-in Custom Agents" section (list the two agents, where they appear, how to disable, how to extend, no specific model required).
2. Update root CHANGELOG.md Unreleased per .github/instructions/changelog_md.instructions.md.
3. Verify all integration + unit tests pass; lint clean.

Relevant files

Modify

• extensions/vscode/package.json — settings, contribution points, scripts.
• extensions/vscode/src/extension.ts — wire the new agent registrar.
• extensions/vscode/.vscodeignore — exclude customizations/ source dir.
• extensions/vscode/README.md — new "Built-in Custom Agents" section.
• CHANGELOG.md — Unreleased entry.

New

• extensions/vscode/customizations/agents/codeql-query-developer.agent.md
• extensions/vscode/customizations/agents/codeql-workshop-author.agent.md
• extensions/vscode/customizations/bundle-customizations.config.js
• extensions/vscode/scripts/bundle-customizations.js
• extensions/vscode/src/customizations/agent-registrar.ts
• extensions/vscode/test/customizations/bundle-customizations.test.ts
• extensions/vscode/test/customizations/agent-registrar.test.ts
• extensions/vscode/test/integration/agents.integration.test.ts (primary suite)
• extensions/vscode/examples/team-customizations/ (sample overlay)

Reference (read-only — do not modify)

• .github/agents/*.md
• server/src/prompts/*.prompt.md
• .github/skills/*/SKILL.md
• extensions/vscode/scripts/bundle-server.js (sibling pattern reference)

Acceptance criteria

• @vscode/test-electron integration tests cover all behaviors in §Implementation methodology and pass green.
• Vitest unit tests for bundler overlay semantics and agent registrar pass green.
• npm run package --workspace=extensions/vscode succeeds and produces a VSIX containing agents/codeql-query-developer.agent.md, agents/codeql-workshop-author.agent.md, the whitelisted prompt + skill files, with customizations/ source dir absent.
• npm run package --workspace=extensions/vscode -- --customizations-dir=./examples/team-customizations produces a VSIX where overlay files replace defaults with collision warnings printed.
• npm run lint --workspace=extensions/vscode passes.
• Neither shipped .agent.md contains a model: key, and no code path forces a specific model.
• Manual smoke: install the VSIX in a fresh VS Code profile, open a workspace, run Chat: Open Customizations — both agents appear as "extension-contributed"; switching to codeql-query-developer and invoking a contributed slash command works; toggling codeql-mcp.agents.enabled to false removes them; setting codeql-mcp.additionalAgentDirs to a temp dir surfaces user files.
• codeql-mcp.showAgentsStatus command prints bundled + extra paths.
• extensions/vscode/README.md and CHANGELOG.md updated.
• All five .github/agents/*.md files in this repo remain unchanged.

Out of scope (track separately)

• Renaming or removing repo-local .github/agents/*.md files.
• Server-side MCP changes (Support user-provided directory of *.prompt.md files #2 already provides --prompts-dir).
• Runtime merging of additionalPromptDirs / additionalSkillDirs (documented contribution points are static — follow-up issue mirroring this design).
• A vscode.chat.createChatParticipant-based @codeql chat participant (a different mechanism from custom agents — consider in a separate issue).

Open considerations

1. contributes.chatAgents availability — not in the published 2026-05 Contribution Points reference. Recommendation: try-then-fallback. If proposed APIs require enabledApiProposals, decide whether to gate behind that or stay programmatic-only.
2. Skill curation — start with three (create-codeql-query-tdd-generic, create-codeql-query-development-workshop, validate-ql-mcp-server-tools-queries); expand based on user feedback.

References

• VS Code Custom Agents docs
• VS Code Chat Participant API
• VS Code Contribution Points (note: chatInstructions, chatPromptFiles, chatSkills documented; chatAgents not yet)
• Related: Support user-provided directory of *.prompt.md files #2 (--prompts-dir user extensibility for MCP server prompts)