Add GitHub Copilot Connection and associated dashboards#8727
Closed
ewega wants to merge 128 commits intoapache:mainfrom
Closed
Add GitHub Copilot Connection and associated dashboards#8727ewega wants to merge 128 commits intoapache:mainfrom
ewega wants to merge 128 commits intoapache:mainfrom
Conversation
Add .specify/ directory with: - memory/constitution.md: Apache DevLake project constitution v1.0.0 - Plugin Independence principle - Three-Layer Data Model (Raw/Tool/Domain) - Test-Driven Development requirements - Migration-First Schema Changes - Apache Compliance requirements - templates/: spec, plan, tasks, checklist, agent-file templates - scripts/powershell/: automation scripts for feature workflows
Add GitHub Copilot agent mode definitions: - .github/agents/: Agent configurations (specify, plan, tasks, etc.) - .github/prompts/: Prompt files linking to agents These enable /speckit.* commands in VS Code Copilot for Spec-Driven Development workflow.
Research for building a GitHub Copilot DevLake plugin: - copilot_api_research.md: API endpoint documentation - copilot_api_actual_responses.md: Live API test results (octodemo org) - copilot_implementation_strategy.md: Option B (repo-level) approach - copilot_plugin_spec.md: Full plugin specification - copilot_research_summary.md: Research findings summary Key decisions: - Org/team-level metrics (not per-user) for privacy - Time-based before/after correlation with DORA metrics - Implementation date configuration for impact analysis - Three-phase MVP approach Ready for Spec-Driven Development implementation.
…tegy documents - Created `copilot_api_actual_responses.md` to document actual API responses and key metrics. - Developed `copilot_api_research.md` summarizing API endpoints, usage metrics, and authentication requirements. - Established `copilot_implementation_strategy.md` outlining the strategy for repository/project-level analysis. - Drafted `copilot_plugin_spec.md` detailing the plugin's specifications, data models, and API endpoints. - Compiled `copilot_research_summary.md` to summarize findings and decisions made during the research phase.
…a model and API specifications
…d error handling strategies
Implements REST endpoints for Copilot connections and scopes, plus a Test Connection helper against the GitHub billing endpoint with friendly error handling.
Adds stateful collectors for /copilot/metrics and /copilot/billing/seats, plus extractors into tool tables with Retry-After handling and unit/e2e coverage.
Marks Phase 4 (US2) tasks as completed now that collectors/extractor, fixtures, and tests are in place.
Adds Copilot Adoption dashboard panels for active/engaged users, acceptance rate, chat usage, and seat timeline based on _tool_copilot_org_metrics.
Updates quickstart with the Copilot Adoption dashboard name/variables and marks US3 tasks complete.
…and enhance Dockerfile for script execution
Support both top-level arrays and wrapped {seats:[...]} responses; mark Copilot subtasks as CROSS domain type.
- Remove azure-deployment.agent.md and consolidate deployment paths into deploy-devlake.agent.md. - Enhance azure-devlake-deployment skill with new deployment paths and detailed instructions for official and custom setups. - Update deploy.ps1 script to support official images and remote repository cloning. - Add main-official.bicep for deploying official images without ACR. - Introduce setup-official.ps1 for easy local setup of DevLake. - Expand troubleshooting guide to address MySQL server issues during Bicep deployment.
…remove infer option
…nd AKS - Create `aci-deployment.md` for deploying DevLake on Azure Container Instances with detailed steps and environment variables. - Create `aks-deployment.md` for deploying DevLake on Azure Kubernetes Service, including cluster creation and manifest usage. - Add `cleanup.md` for resource cleanup instructions, including quick and selective cleanup options. - Introduce `cli-commands.md` as a reference for Azure CLI commands related to deployment and management. - Add `environment-variables.md` to document required and recommended environment variables for DevLake. - Create `k8s-manifest.yaml` for Kubernetes deployment configuration, including services and secrets. - Add `powershell-escaping.md` to address PowerShell escaping issues for environment variables. - Introduce `troubleshooting.md` for common issues and solutions during deployment and operation. - Add `setup-official.ps1` script for downloading and setting up Apache DevLake using Docker Compose.
…nt (#21) Add GitHub Copilot custom agent for Azure deployment workflows with MCP tools integration
Migrate the GitHub Copilot plugin from the deprecated legacy metrics API to the new Copilot Usage Metrics Reports API (report-download flow). Backend changes: - Add enterprise-level and user-level metric collectors using the report metadata + signed-URL download pattern - Add new models: enterprise daily metrics, user daily metrics, and breakdown tables (by IDE, feature, language, model) - Add enterprise slug support to connection and scope models - Add migration scripts for the v2 schema - Rewrite org metrics extractor for new flat report format - Handle enterprise seat collection via /billing/seats endpoint - Handle Azure blob 404s gracefully in report downloads Frontend changes: - Add enterprise slug field to GitHub Copilot connection config Dashboard changes: - Revamp Copilot Adoption dashboard with expanded KPI/trend/mix/ quality/user-seat/diagnostics panels - Redesign Feature Mix and IDE Adoption panels as stacked bar charts - Fix Impact dashboard SQL to use connection-aware seat denominators and proper Grafana variable interpolation for project filtering Housekeeping: - Add playwright/copilot/claude artifacts to .gitignore Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Rewrite Feature Mix Over Time and IDE Adoption Over Time panels (plus their 100% share variants) to properly display stacked bar charts with distinct colors per category. Changes: - Use long-form SQL (time, metric, value) with Grafana prepareTimeSeries + renameByRegex transformations to create named series per category - Enable stacking.mode normal (absolute) and percent (share panels) - Clean feature names by stripping chat_panel_ prefix and _mode suffix - Reposition share panels directly below absolute panels for comparison - Dynamic top-4 categories via ROW_NUMBER window function + Other bucket Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- WAU/MAU: New API's 1-day report doesn't include weekly/monthly active users. Compute from user-level data (COUNT DISTINCT user_login over trailing 7/28 days from _tool_copilot_user_daily_metrics). - LOC Yield: Remove percentunit format and max:1 cap since LOC yield is a ratio (typically >1x), not a percentage. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- All 27 panels now query both _tool_copilot_enterprise_daily_metrics and _tool_copilot_org_daily_metrics via UNION ALL, so the dashboard works regardless of whether the connection is enterprise or org level. - Fixed ONLY_FULL_GROUP_BY error caused by non-aggregated columns in the outer GROUP BY of the unified CTE. - Fixed Current Adoption % (panel 5): was summing DAU across all days in the 7-day window then dividing by seats (showed 127%); now uses latest day's DAU for correct percentage (33.8%). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The org-level connection (connection_id=1) has no seat data, making adoption % calculations return NULL. Changed the connection_id variable query to sort DESC so the enterprise connection (which has full seat and adoption data) is selected by default on both dashboards. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Set min:0, max:100 on Adoption % override for panels: Adoption vs Deployment Frequency, Change Failure Rate, MTTR, and Review Time Trend - Rename dashboard title from 'GitHub Copilot Impact' to 'GitHub Copilot DORA Correlation' - Update intro panel text to match new name Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Gh copilot connection migrate
…reSave signatures - Add missing scope-config/:scopeConfigId/projects route and handler (config-ui calls this when editing a scope config) - Fix BeforeSave signatures to match GORM v2 interface (*gorm.DB) - Fix ScopeDetail type alias to use GhCopilotScopeConfig instead of NoScopeConfig
…-connection Bugfix edit scope config gh copilot connection
Poetry 2.x dropped support for Python 3.9. Since the Dockerfile uses python:3.9-slim-bookworm, the Poetry install fails with 2.3.2+. Pin POETRY_VERSION=1.8.5 to restore the build.
TokenExpiresAt and RefreshTokenExpiresAt are *time.Time but BeforeSave was assigning time.Time values directly, causing compile error. Also added nil checks before calling IsZero() to prevent nil dereference.
…, fix Grafana scope_id - Add Enterprise Slug input to Config-UI connection form - Update Organization field help text to reference Enterprise - Unify org + enterprise metrics into shared by-language, by-ide, by-feature, and by-model tables in the extractor - Fix Grafana scope_id template variable to query _tool_copilot_scopes instead of _tool_copilot_enterprise_daily_metrics
Repo transferred from ewega/incubator-devlake to DevExpGBB/incubator-devlake. Updated all example URLs and references in skills and agent docs.
…lot connection Organization is no longer individually required. At least one of Organization or Enterprise Slug must be provided. Shared error state via scopeIdentity key. New styled ErrorText component for inline errors.
4 tasks
Contributor
Author
Contributor
Author
|
Closing in favor of #8728 (clean branch/history). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
pr-type/bug-fix,pr-type/feature-development, etc.Summary
What does this PR do?
Does this close any open issues?
Closes xx
Screenshots
Include any relevant screenshots here.
Other Information
Any other information that is important to this PR.