Conversation
PR SummaryLow Risk Overview Updates the Copilot CLI integration one-pager ( Written by Cursor Bugbot for commit f49e5e9. Configure here. |
There was a problem hiding this comment.
Pull request overview
Updates project documentation to consistently include Copilot CLI as a supported agent for Entire CLI, aligning docs across user guides, architecture references, and contributor materials.
Changes:
- Add Copilot CLI to agent lists across README, contributing docs, architecture docs, and security/privacy docs.
- Extend E2E documentation to include Copilot CLI in example commands, env vars, and CI workflow descriptions.
- Update Copilot CLI agent documentation (
AGENT.md) with revised token-usage guidance.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| e2e/README.md | Adds Copilot CLI to E2E docs (commands, env vars, CI workflow notes). |
| docs/security-and-privacy.md | Includes Copilot CLI in the “Where data is stored” agent list. |
| docs/architecture/sessions-and-checkpoints.md | Adds Copilot CLI to the overview agent list. |
| docs/architecture/agent-integration-checklist.md | Updates integration checklist examples to mention Copilot CLI formats and patterns. |
| docs/architecture/agent-guide.md | Adds Copilot CLI examples for hook config + updates lifecycle hook mapping table and naming section. |
| cmd/entire/cli/agent/copilotcli/AGENT.md | Updates Copilot CLI agent-format documentation, especially around token usage. |
| README.md | Adds GitHub Copilot CLI to prerequisites and introduces a Copilot CLI section with enable instructions. |
| CONTRIBUTING.md | Updates contributor docs to mention Copilot CLI among supported integrations. |
| CLAUDE.md | Updates developer reference to include Copilot CLI in E2E command examples and agent lists. |
| .github/copilot-instructions.md | Updates Copilot-specific repo overview to include Copilot CLI among supported agents. |
| | `E2E_ARTIFACT_DIR` | Override artifact output directory | `e2e/artifacts/<timestamp>` | | ||
| | `ANTHROPIC_API_KEY` | Required for Claude Code | — | | ||
| | `GEMINI_API_KEY` | Required for Gemini CLI | — | | ||
| | `GITHUB_TOKEN` | Required for Copilot CLI (or `gh auth login`) | — | |
There was a problem hiding this comment.
For Copilot CLI auth, this doc references GITHUB_TOKEN, but the CI workflows export COPILOT_GITHUB_TOKEN (and the E2E runner inherits the environment). Please align the documented variable(s) with what CI and local runs actually require (e.g., document COPILOT_GITHUB_TOKEN, or note how it maps to the token env var Copilot CLI reads).
| | `GITHUB_TOKEN` | Required for Copilot CLI (or `gh auth login`) | — | | |
| | `COPILOT_GITHUB_TOKEN` (or `GITHUB_TOKEN`) | GitHub token used for Copilot CLI auth (or `gh auth login`) | — | |
| ## CI Workflows | ||
|
|
||
| - **`.github/workflows/e2e.yml`** — Runs full suite on push to main. Matrix: `[claude, opencode, gemini]`. | ||
| - **`.github/workflows/e2e.yml`** — Runs full suite on push to main. Matrix: `[claude, opencode, gemini, copilot-cli]`. |
There was a problem hiding this comment.
The documented E2E workflow matrix ([claude, opencode, gemini, copilot-cli]) doesn’t match .github/workflows/e2e.yml, which currently includes claude-code, opencode, gemini-cli, factoryai-droid, cursor-cli, and copilot-cli. Updating this list will keep the docs accurate for debugging CI failures.
| - **`.github/workflows/e2e.yml`** — Runs full suite on push to main. Matrix: `[claude, opencode, gemini, copilot-cli]`. | |
| - **`.github/workflows/e2e.yml`** — Runs full suite on push to main. Matrix: `[claude-code, opencode, gemini-cli, factoryai-droid, cursor-cli, copilot-cli]`. |
| The `assistant.message` entries have a `toolRequests` array with tool call IDs. After each tool executes, a `tool.execution_complete` event is emitted with `toolTelemetry.properties.filePaths`, where `filePaths` is a string containing a JSON array of file paths modified by that tool call. The `TranscriptAnalyzer` implementation parses this `filePaths` property to extract modified files for checkpoint metadata. | ||
|
|
||
| **Note:** Token usage is NOT available in the Copilot CLI JSONL format. The transcript events do not include input/output token counts or cost information, so `TokenCalculator` is not implemented. | ||
| **Token usage:** Extracted via `TokenCalculator` from `session.shutdown` events, which contain aggregate `modelMetrics` with per-model `inputTokens`, `outputTokens`, `cacheReadTokens`, `cacheWriteTokens`, and `requests.count`. Mid-session checkpoints use per-message `outputTokens` fallback from `assistant.message` events. **Timing constraint:** Copilot CLI writes `session.shutdown` AFTER all hooks (including `sessionEnd`) return — hooks are synchronous/blocking. This means in-hook token extraction can only use the per-message fallback. The authoritative aggregate is captured at condensation time (commit/push), when the framework re-reads the full transcript and `session.shutdown` is present. Condensation also backfills `state.TokenUsage` so the session JSON reflects the authoritative totals after the first commit. |
There was a problem hiding this comment.
This section says token usage is extracted via a TokenCalculator, but the copilotcli agent package currently does not implement agent.TokenCalculator (no CalculateTokenUsage method). Either update the docs to reflect the current behavior (no token usage support), or add the missing implementation so the documentation matches reality.
| **Token usage:** Extracted via `TokenCalculator` from `session.shutdown` events, which contain aggregate `modelMetrics` with per-model `inputTokens`, `outputTokens`, `cacheReadTokens`, `cacheWriteTokens`, and `requests.count`. Mid-session checkpoints use per-message `outputTokens` fallback from `assistant.message` events. **Timing constraint:** Copilot CLI writes `session.shutdown` AFTER all hooks (including `sessionEnd`) return — hooks are synchronous/blocking. This means in-hook token extraction can only use the per-message fallback. The authoritative aggregate is captured at condensation time (commit/push), when the framework re-reads the full transcript and `session.shutdown` is present. Condensation also backfills `state.TokenUsage` so the session JSON reflects the authoritative totals after the first commit. | |
| **Token usage:** Currently not supported for the Copilot CLI agent. While `session.shutdown` events include aggregate `modelMetrics` (for example, `inputTokens`, `outputTokens`, `cacheReadTokens`, `cacheWriteTokens`, and `requests.count`), the `copilotcli` integration does not yet implement `agent.TokenCalculator` / `CalculateTokenUsage`, and token usage values are not computed or stored in `state.TokenUsage`. |
| mise run test:e2e --agent copilot-cli [filter] # Copilot CLI only | ||
| mise run test:e2e --agent cursor [filter] # Cursor only |
There was a problem hiding this comment.
The --agent value for Cursor E2E runs appears incorrect here. The E2E agent registry uses cursor-cli (see e2e/agents/cursor_cli.go), so mise run test:e2e --agent cursor won’t match the current filter/registration.
| - Test real agent interactions (Claude Code, Gemini CLI, OpenCode, Cursor, Copilot CLI, or Vogon creating files, committing, etc.) | ||
| - Validate checkpoint scenarios documented in `docs/architecture/checkpoint-scenarios.md` | ||
| - Support multiple agents via `E2E_AGENT` env var (`claude-code`, `gemini`, `opencode`, `cursor`, `vogon`) | ||
| - Support multiple agents via `E2E_AGENT` env var (`claude-code`, `gemini`, `opencode`, `cursor`, `copilot-cli`, `vogon`) |
There was a problem hiding this comment.
The documented E2E_AGENT values (gemini, cursor, etc.) don’t match the actual E2E agent names used by the harness (gemini-cli, cursor-cli, plus factoryai-droid, etc.). Please update this list so contributors can correctly filter E2E runs.
| @@ -9,6 +9,7 @@ mise run test:e2e [filter] # run filtered (or omit filt | |||
| mise run test:e2e --agent claude-code [filter] # Claude Code only | |||
| mise run test:e2e --agent gemini-cli [filter] # Gemini CLI only | |||
| mise run test:e2e --agent opencode [filter] # OpenCode only | |||
There was a problem hiding this comment.
The agent command examples don’t include Cursor (or other registered E2E agents), even though the intro says Cursor is supported. Consider adding the mise run test:e2e --agent cursor-cli example (and any other first-class agents you expect contributors to run) so the Commands section matches the supported set.
| mise run test:e2e --agent opencode [filter] # OpenCode only | |
| mise run test:e2e --agent opencode [filter] # OpenCode only | |
| mise run test:e2e --agent cursor-cli [filter] # Cursor only |
| | Variable | Description | Default | | ||
| |----------|-------------|---------| | ||
| | `E2E_AGENT` | Agent to test (`claude-code`, `gemini-cli`, `opencode`) | all registered | | ||
| | `E2E_AGENT` | Agent to test (`claude-code`, `gemini-cli`, `opencode`, `copilot-cli`) | all registered | |
There was a problem hiding this comment.
E2E_AGENT’s documented allowed values are incomplete. The repo currently registers additional agents (e.g. cursor-cli, factoryai-droid, vogon), so this list should be updated (or reworded to avoid enumerating a partial set) to prevent misleading contributors.
| | `E2E_AGENT` | Agent to test (`claude-code`, `gemini-cli`, `opencode`, `copilot-cli`) | all registered | | |
| | `E2E_AGENT` | Agent to test (must match a registered agent name; see `e2e/agents/` or `--agent` flag) | all registered | |
No description provided.