Skip to content

docs: add Copilot CLI to all documentation#653

Draft
peyton-alt wants to merge 1 commit intomainfrom
docs/add-copilot-cli
Draft

docs: add Copilot CLI to all documentation#653
peyton-alt wants to merge 1 commit intomainfrom
docs/add-copilot-cli

Conversation

@peyton-alt
Copy link
Contributor

No description provided.

Copilot AI review requested due to automatic review settings March 7, 2026 00:53
@cursor
Copy link

cursor bot commented Mar 7, 2026

PR Summary

Low Risk
Documentation-only changes; no runtime code or behavior changes, aside from low-risk potential for minor doc mismatches.

Overview
Documents GitHub Copilot CLI as a first-class supported agent across repo docs (README, contributing/architecture guides, security/privacy, Copilot instructions), including the entire enable --agent copilot-cli flow, hook config location (.github/hooks/entire.json), and E2E test invocations/env vars.

Updates the Copilot CLI integration one-pager (cmd/entire/cli/agent/copilotcli/AGENT.md) to describe how token usage is derived (per-message fallback vs authoritative totals from session.shutdown available only during condensation).

Written by Cursor Bugbot for commit f49e5e9. Configure here.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates project documentation to consistently include Copilot CLI as a supported agent for Entire CLI, aligning docs across user guides, architecture references, and contributor materials.

Changes:

  • Add Copilot CLI to agent lists across README, contributing docs, architecture docs, and security/privacy docs.
  • Extend E2E documentation to include Copilot CLI in example commands, env vars, and CI workflow descriptions.
  • Update Copilot CLI agent documentation (AGENT.md) with revised token-usage guidance.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
e2e/README.md Adds Copilot CLI to E2E docs (commands, env vars, CI workflow notes).
docs/security-and-privacy.md Includes Copilot CLI in the “Where data is stored” agent list.
docs/architecture/sessions-and-checkpoints.md Adds Copilot CLI to the overview agent list.
docs/architecture/agent-integration-checklist.md Updates integration checklist examples to mention Copilot CLI formats and patterns.
docs/architecture/agent-guide.md Adds Copilot CLI examples for hook config + updates lifecycle hook mapping table and naming section.
cmd/entire/cli/agent/copilotcli/AGENT.md Updates Copilot CLI agent-format documentation, especially around token usage.
README.md Adds GitHub Copilot CLI to prerequisites and introduces a Copilot CLI section with enable instructions.
CONTRIBUTING.md Updates contributor docs to mention Copilot CLI among supported integrations.
CLAUDE.md Updates developer reference to include Copilot CLI in E2E command examples and agent lists.
.github/copilot-instructions.md Updates Copilot-specific repo overview to include Copilot CLI among supported agents.

| `E2E_ARTIFACT_DIR` | Override artifact output directory | `e2e/artifacts/<timestamp>` |
| `ANTHROPIC_API_KEY` | Required for Claude Code | — |
| `GEMINI_API_KEY` | Required for Gemini CLI | — |
| `GITHUB_TOKEN` | Required for Copilot CLI (or `gh auth login`) | — |
Copy link

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For Copilot CLI auth, this doc references GITHUB_TOKEN, but the CI workflows export COPILOT_GITHUB_TOKEN (and the E2E runner inherits the environment). Please align the documented variable(s) with what CI and local runs actually require (e.g., document COPILOT_GITHUB_TOKEN, or note how it maps to the token env var Copilot CLI reads).

Suggested change
| `GITHUB_TOKEN` | Required for Copilot CLI (or `gh auth login`) ||
| `COPILOT_GITHUB_TOKEN` (or `GITHUB_TOKEN`) | GitHub token used for Copilot CLI auth (or `gh auth login`) ||

Copilot uses AI. Check for mistakes.
## CI Workflows

- **`.github/workflows/e2e.yml`** — Runs full suite on push to main. Matrix: `[claude, opencode, gemini]`.
- **`.github/workflows/e2e.yml`** — Runs full suite on push to main. Matrix: `[claude, opencode, gemini, copilot-cli]`.
Copy link

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documented E2E workflow matrix ([claude, opencode, gemini, copilot-cli]) doesn’t match .github/workflows/e2e.yml, which currently includes claude-code, opencode, gemini-cli, factoryai-droid, cursor-cli, and copilot-cli. Updating this list will keep the docs accurate for debugging CI failures.

Suggested change
- **`.github/workflows/e2e.yml`** — Runs full suite on push to main. Matrix: `[claude, opencode, gemini, copilot-cli]`.
- **`.github/workflows/e2e.yml`** — Runs full suite on push to main. Matrix: `[claude-code, opencode, gemini-cli, factoryai-droid, cursor-cli, copilot-cli]`.

Copilot uses AI. Check for mistakes.
The `assistant.message` entries have a `toolRequests` array with tool call IDs. After each tool executes, a `tool.execution_complete` event is emitted with `toolTelemetry.properties.filePaths`, where `filePaths` is a string containing a JSON array of file paths modified by that tool call. The `TranscriptAnalyzer` implementation parses this `filePaths` property to extract modified files for checkpoint metadata.

**Note:** Token usage is NOT available in the Copilot CLI JSONL format. The transcript events do not include input/output token counts or cost information, so `TokenCalculator` is not implemented.
**Token usage:** Extracted via `TokenCalculator` from `session.shutdown` events, which contain aggregate `modelMetrics` with per-model `inputTokens`, `outputTokens`, `cacheReadTokens`, `cacheWriteTokens`, and `requests.count`. Mid-session checkpoints use per-message `outputTokens` fallback from `assistant.message` events. **Timing constraint:** Copilot CLI writes `session.shutdown` AFTER all hooks (including `sessionEnd`) return — hooks are synchronous/blocking. This means in-hook token extraction can only use the per-message fallback. The authoritative aggregate is captured at condensation time (commit/push), when the framework re-reads the full transcript and `session.shutdown` is present. Condensation also backfills `state.TokenUsage` so the session JSON reflects the authoritative totals after the first commit.
Copy link

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section says token usage is extracted via a TokenCalculator, but the copilotcli agent package currently does not implement agent.TokenCalculator (no CalculateTokenUsage method). Either update the docs to reflect the current behavior (no token usage support), or add the missing implementation so the documentation matches reality.

Suggested change
**Token usage:** Extracted via `TokenCalculator` from `session.shutdown` events, which contain aggregate `modelMetrics` with per-model `inputTokens`, `outputTokens`, `cacheReadTokens`, `cacheWriteTokens`, and `requests.count`. Mid-session checkpoints use per-message `outputTokens` fallback from `assistant.message` events. **Timing constraint:** Copilot CLI writes `session.shutdown` AFTER all hooks (including `sessionEnd`) return — hooks are synchronous/blocking. This means in-hook token extraction can only use the per-message fallback. The authoritative aggregate is captured at condensation time (commit/push), when the framework re-reads the full transcript and `session.shutdown` is present. Condensation also backfills `state.TokenUsage` so the session JSON reflects the authoritative totals after the first commit.
**Token usage:** Currently not supported for the Copilot CLI agent. While `session.shutdown` events include aggregate `modelMetrics` (for example, `inputTokens`, `outputTokens`, `cacheReadTokens`, `cacheWriteTokens`, and `requests.count`), the `copilotcli` integration does not yet implement `agent.TokenCalculator` / `CalculateTokenUsage`, and token usage values are not computed or stored in `state.TokenUsage`.

Copilot uses AI. Check for mistakes.
Comment on lines +75 to +76
mise run test:e2e --agent copilot-cli [filter] # Copilot CLI only
mise run test:e2e --agent cursor [filter] # Cursor only
Copy link

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The --agent value for Cursor E2E runs appears incorrect here. The E2E agent registry uses cursor-cli (see e2e/agents/cursor_cli.go), so mise run test:e2e --agent cursor won’t match the current filter/registration.

Copilot uses AI. Check for mistakes.
Comment on lines +84 to +86
- Test real agent interactions (Claude Code, Gemini CLI, OpenCode, Cursor, Copilot CLI, or Vogon creating files, committing, etc.)
- Validate checkpoint scenarios documented in `docs/architecture/checkpoint-scenarios.md`
- Support multiple agents via `E2E_AGENT` env var (`claude-code`, `gemini`, `opencode`, `cursor`, `vogon`)
- Support multiple agents via `E2E_AGENT` env var (`claude-code`, `gemini`, `opencode`, `cursor`, `copilot-cli`, `vogon`)
Copy link

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documented E2E_AGENT values (gemini, cursor, etc.) don’t match the actual E2E agent names used by the harness (gemini-cli, cursor-cli, plus factoryai-droid, etc.). Please update this list so contributors can correctly filter E2E runs.

Copilot uses AI. Check for mistakes.
@@ -9,6 +9,7 @@ mise run test:e2e [filter] # run filtered (or omit filt
mise run test:e2e --agent claude-code [filter] # Claude Code only
mise run test:e2e --agent gemini-cli [filter] # Gemini CLI only
mise run test:e2e --agent opencode [filter] # OpenCode only
Copy link

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The agent command examples don’t include Cursor (or other registered E2E agents), even though the intro says Cursor is supported. Consider adding the mise run test:e2e --agent cursor-cli example (and any other first-class agents you expect contributors to run) so the Commands section matches the supported set.

Suggested change
mise run test:e2e --agent opencode [filter] # OpenCode only
mise run test:e2e --agent opencode [filter] # OpenCode only
mise run test:e2e --agent cursor-cli [filter] # Cursor only

Copilot uses AI. Check for mistakes.
| Variable | Description | Default |
|----------|-------------|---------|
| `E2E_AGENT` | Agent to test (`claude-code`, `gemini-cli`, `opencode`) | all registered |
| `E2E_AGENT` | Agent to test (`claude-code`, `gemini-cli`, `opencode`, `copilot-cli`) | all registered |
Copy link

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

E2E_AGENT’s documented allowed values are incomplete. The repo currently registers additional agents (e.g. cursor-cli, factoryai-droid, vogon), so this list should be updated (or reworded to avoid enumerating a partial set) to prevent misleading contributors.

Suggested change
| `E2E_AGENT` | Agent to test (`claude-code`, `gemini-cli`, `opencode`, `copilot-cli`) | all registered |
| `E2E_AGENT` | Agent to test (must match a registered agent name; see `e2e/agents/` or `--agent` flag) | all registered |

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants