feat(compile): runtime prompt injection via prompt.js bundle

[jamesadevine]

Summary

Replaces compile-time prompt embedding with runtime prompt injection via a new prompt.js ado-script bundle. The default behaviour changes: the agent's markdown body is no longer baked into the compiled pipeline YAML — it's read from the workspace at runtime by prompt.js, which also strips front matter, appends extension supplements, and substitutes pipeline parameters / variables.

UX win: body-only edits to the agent .md no longer require recompiling the pipeline. Front-matter changes still recompile (correct: front-matter is configuration, body is prompt).

Set inlined-imports: true in front matter (mirrors gh-aw's field name exactly) to opt out and keep the legacy heredoc-embedded behaviour. Use this for self-contained YAML, restricted networks, or debugging.

Design

The runtime contract is a new PromptSpec IR mirroring the proven GateSpec pattern:

• src/compile/prompt_ir.rs — PromptSpec, PromptSupplement, PROMPT_SPEC_VERSION = 1, schemars-derived JSON Schema.
• Hidden CLI subcommand export-prompt-schema (mirrors export-gate-schema).
• npm run codegen now produces both types.gen.ts (gate) and types-prompt.gen.ts (prompt). CI drift check covers both.
• PromptSpec is JSON-serialized + base64-encoded into a single ADO_AW_PROMPT_SPEC env var on the prompt.js step. Each declared parameter gets a separate ADO_AW_PARAM_<NAME>: ${{ parameters.<NAME> }} env entry — values stay outside the spec so ADO secret-redaction and late binding work correctly.

Substitution patterns recognised by prompt.js

Pattern	Resolved via	Notes
${{ parameters.NAME }}	env ADO_AW_PARAM_<NAME upper, hyphen→underscore>	Only declared parameters substitute; others left verbatim with a warning.
$(VAR) / $(VAR.SUB)	env <name upper, dot→underscore> (ADO native)	Unset variables left verbatim with a warning. Secrets aren't auto-exposed and stay verbatim.
$[ ... ]	not substituted	Left verbatim with one warning per render.
\$(...)	escape	Backslash stripped; $(...) left literal.

Pipeline ordering

When inlined-imports: false (default), the new {{ prepare_agent_prompt }} template marker emits a three-step bundle in the Agent job:

1. NodeTool@0 to install Node 20.x.
2. curl download of scripts.zip and unzip to /tmp/ado-aw-scripts/ (factored into a shared helper used by both gate.js Setup-job step and prompt.js Agent-job step).
3. bash: node /tmp/ado-aw-scripts/prompt.js with the spec env + parameter env mappings.

When inlined-imports: true, the same marker emits the legacy heredoc step embedding the body verbatim, plus per-extension cat >> supplement steps via the existing wrap_prompt_append.

Scope

• ✅ Body injection from workspace .md.
• ✅ Extension prompt supplements consolidated in PromptSpec.supplements.
• ✅ Variable substitution for ${{ parameters.* }} and $(VAR) patterns.
• ❌ {{#import path }} runtime-import macros — out of scope; the spec is additive-friendly so this is not a one-way door.
• ❌ Threat-analysis prompt stays compile-time embedded — it's compiler-internal, not user-visible.

Files changed

Rust (compile-time)

• src/compile/prompt_ir.rs (new) — PromptSpec, PromptSupplement, PROMPT_SPEC_VERSION, schema generator.
• src/compile/mod.rs — pub(crate) mod prompt_ir;.
• src/compile/types.rs — adds inlined_imports: bool (#[serde(rename = "inlined-imports", default)]) to FrontMatter.
• src/compile/common.rs — new helpers collect_prompt_supplements, generate_prepare_agent_prompt; generate_prepare_steps takes inlined_imports: bool and skips wrap_prompt_append in the runtime branch; compile_shared builds and emits {{ prepare_agent_prompt }}.
• src/compile/extensions/mod.rs — extracts scripts_download_step() and node_tool_step(name) helpers shared by both bundles.
• src/compile/extensions/trigger_filters.rs — uses the shared helpers.
• src/main.rs — adds hidden Commands::ExportPromptSchema subcommand.
• src/data/base.yml and src/data/1es-base.yml — replace the static heredoc step with {{ prepare_agent_prompt }}.

TypeScript (runtime bundle)

• scripts/ado-script/src/prompt/index.ts (new) — entry point; decodes ADO_AW_PROMPT_SPEC, validates version, reads source, strips front matter, substitutes, appends supplements, writes output.
• scripts/ado-script/src/prompt/frontmatter.ts (new) — pure stripFrontMatter mirroring extract_front_matter in Rust.
• scripts/ado-script/src/prompt/substitute.ts (new) — substitution engine (${{ parameters.* }}, $(VAR), \$(...), $[...] warning).
• scripts/ado-script/src/prompt/__tests__/{frontmatter,substitute}.test.ts (new) — 19 unit tests.
• scripts/ado-script/src/shared/types-prompt.gen.ts (new, AUTO-GENERATED) — committed; CI verifies it stays in lockstep with the Rust IR.
• scripts/ado-script/test/prompt-smoke.test.ts (new) — 3 end-to-end smoke tests against the compiled bundle.
• scripts/ado-script/package.json — extends codegen to also produce the prompt schema; adds build:prompt; updates build and build:check to cover both bundles.

CI / infra

• .github/workflows/ado-script.yml — drift check now covers both types.gen.ts and types-prompt.gen.ts.
• .github/workflows/release.yml — comment updated; no zip-command change needed (the existing glob picks up scripts/prompt.js automatically).
• .gitignore — adds scripts/prompt.js.

Docs

• docs/ado-script.md — adds prompt.js bundle section, updated architecture diagram, codegen pipeline now produces both schemas.
• docs/front-matter.md — documents inlined-imports: true.
• docs/template-markers.md — replaces {{ agent_content }} with {{ prepare_agent_prompt }} and documents the substitution contract.
• docs/extending.md — notes that prompt_supplement() content goes through the substitution pipeline at runtime.
• AGENTS.md — adds prompt.js and prompt_ir.rs to the source tree overview.

Test plan

• cargo build ✓
• cargo test — 1117 + 78 + others pass; 8 new unit tests in compile::common covering both branches, both call paths through generate_prepare_steps, parameter env emission, supplement ordering, and the source-path validation guard. 3 new tests in compile::types for the inlined-imports field. 4 new tests in compile::prompt_ir for the schema and serialization.
• cargo clippy --all-targets --all-features — no new lints (baseline pre-existed; verified by stash-pop comparison).
• cd scripts/ado-script && npm test — 191 tests pass (172 baseline + 19 new for prompt.js: 8 frontmatter + 11 substitute).
• cd scripts/ado-script && npm run typecheck — clean.
• cd scripts/ado-script && npm run test:smoke — 5 tests pass (gate.js smoke + 3 new prompt.js end-to-end tests covering happy path, missing source, and unknown version).
• Manual end-to-end spot-check:
  • Default branch: compiled examples/sample-agent.md and verified the YAML emits node /tmp/ado-aw-scripts/prompt.js + ADO_AW_PROMPT_SPEC env, and that the prompt body is absent from the YAML. Decoded the base64 spec and confirmed source_path resolves to $(Build.SourcesDirectory)/examples/sample-agent.md (i.e., {{ trigger_repo_directory }} is substituted before encoding so prompt.js sees a fully resolved path).
  • Inlined branch: with inlined-imports: true, verified the YAML contains the body verbatim inside an AGENT_PROMPT_EOF heredoc and does not invoke prompt.js.

Migration

Default behaviour changes. Existing compiled .lock.yml files will fail ado-aw check after this lands until the consumer recompiles. The integrity check property strengthens: with the body no longer embedded, prose-only edits no longer drift from the compiled YAML at all.

inlined-imports: true is the one-line escape hatch for users who can't recompile immediately, can't reach github.com from the Agent pool, or want a fully self-contained pipeline file.

[github-actions]

🔍 Rust PR Review

Summary: Looks good with two issues worth addressing — a subtle string-trimming bug in the inlined branch and a double-substitution vector in the TypeScript substitution pipeline.

Findings

🐛 Bugs / Logic Issues

• src/compile/common.rs — trim_start_matches over-strips leading whitespace

  let trimmed = body_indented.trim_start_matches(' ');

  trim_start_matches(' ') strips all leading space characters from the whole string, not just the 4 spaces added by body_indented. For any body whose first line starts with original leading whitespace (e.g., an indented code block as the very first line of the prompt), those spaces are silently swallowed. The intended operation is strip_prefix(" "):

  let trimmed = body_indented.strip_prefix("    ").unwrap_or(&body_indented);

  This is low-impact in practice (agent body first lines are almost always headers or plain text), but it's semantically incorrect and would be surprising for users who carefully indent their prompt openings.

🔒 Security Concerns

• scripts/ado-script/src/prompt/substitute.ts — double-substitution chain

  ${{ parameters.NAME }} substitution runs in step 2 before $(VAR) substitution in step 3. If a pipeline-queued parameter value itself contains an ADO variable pattern — e.g., someone queues with target = $(System.AccessToken) — step 2 injects that literal string into the prompt source, and step 3 then resolves it against the environment. Since ADO_AW_PARAM_* env vars are set from ${{ parameters.NAME }} in the pipeline YAML, ADO expands the parameter value before it lands in the env var, so the env var holds the caller's literal string and step 3 would expand it if that env var exists.

  The Agent-stage token is read-only and the Detection stage is a mitigation, but the behaviour is surprising and worth a comment or a deliberate ordering decision. One option is to run $(VAR) expansion first so parameter values cannot chain into it; another is to document the intentional chain.

⚠️ Suggestions

• src/compile/common.rs — no integration fixture for the runtime compile path

  The PR adds solid unit tests for generate_prepare_agent_prompt and collect_prompt_supplements in #[cfg(test)], but tests/compiler_tests.rs has no new fixture exercising a full compile_shared round-trip with inlined-imports: false (the new default). A snapshot test verifying that {{ prepare_agent_prompt }} is absent from the output YAML (replaced by the three-step bundle) and that ADO_AW_PROMPT_SPEC is present would catch any regression in the template-substitution wiring.

✅ What Looks Good

• PromptSpec IR design mirrors GateSpec cleanly; the base64 envelope correctly isolates the spec from YAML parsing and template substitution.
• replace_with_indent properly propagates the placeholder's line indentation to every line of the multi-step replacement, so both base.yml (6-space) and 1es-base.yml (16-space) render correctly without template-specific logic.
• Version gate in prompt.js (spec.version !== SUPPORTED_VERSION) is tight and provides an actionable error message pointing at a matching release.
• source_path guard (starts_with("$(Build.SourcesDirectory)")) with a descriptive bail! message and an explicit escape hatch (inlined-imports: true) is excellent UX.
• TypeScript test coverage (19 unit tests + 3 smoke tests) is thorough; the beforeEach/afterEach env restoration pattern in substitute.test.ts is correct.
• inlined_imports field in FrontMatter has the right #[serde(rename = "inlined-imports", default)] shape and three clear round-trip tests.

Generated by Rust PR Reviewer for issue #395 · ● 1.8M · ◷