Skip to content

feat(ai,ui): unify cost tracking and chart type taxonomy#401

Merged
hotlong merged 4 commits intomainfrom
copilot/unify-ai-cost-tracking
Jan 30, 2026
Merged

feat(ai,ui): unify cost tracking and chart type taxonomy#401
hotlong merged 4 commits intomainfrom
copilot/unify-ai-cost-tracking

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Jan 30, 2026

Problem

Cost tracking fragmented across AI protocols (conversation, orchestration, NLQ, RAG, predictive). Dashboard declares 27 chart types, Report declares 7 - no shared taxonomy.

Changes

AI Cost Tracking Unification

Created standardized schemas:

  • TokenUsageSchema: {prompt, completion, total}
  • AIOperationCostSchema: operation type, model ID, tokens, cost, timestamp

Integrated into 6 AI protocols:

  • conversation.zod.ts: Added tokens and cost to messages and sessions
  • orchestration.zod.ts: Added to execution results
  • nlq.zod.ts, predictive.zod.ts, rag-pipeline.zod.ts: Added to response schemas
// Before: fragmented token tracking
tokens: z.object({
  prompt: z.number().optional(),
  completion: z.number().optional(),
  total: z.number().optional(),
})

// After: unified schema
import { TokenUsageSchema } from './cost.zod';
tokens: TokenUsageSchema.optional()

Chart Type Taxonomy

Created ui/chart.zod.ts with 27 chart types organized by purpose:

  • Comparison: bar, horizontal-bar, column, grouped-bar, stacked-bar
  • Trend: line, area, stacked-area, step-line
  • Distribution: pie, donut, funnel
  • Relationship: scatter, bubble
  • Composition: treemap, sunburst, sankey
  • Performance: gauge, metric, kpi
  • Geo: choropleth, bubble-map
  • Advanced: heatmap, radar, waterfall, box-plot, violin
  • Tabular: table, pivot

Updated both dashboard.zod.ts and report.zod.ts to import from shared ChartConfigSchema:

// Before: Dashboard had ChartType enum, Report had inline enum
// After: Both use ChartTypeSchema from chart.zod.ts
import { ChartTypeSchema, ChartConfigSchema } from './chart.zod';

export const DashboardWidgetSchema = z.object({
  type: ChartTypeSchema.default('metric'),
  chartConfig: ChartConfigSchema.optional(),
  // ...
});

export const ReportChartSchema = ChartConfigSchema.extend({
  xAxis: z.string(),
  yAxis: z.string(),
  groupBy: z.string().optional(),
});

Impact

  • Single source of truth for AI operation costs
  • Consistent chart types across Dashboard and Report widgets
  • Type-safe cost and token tracking with runtime validation
  • Added 56 tests (35 cost, 21 chart)
Original prompt

Week 3: AI & Data Layer

Task 3.1: Unify AI Cost Tracking 🤖

统一 AI 成本追踪

Issue: Cost tracking fragmented across AI protocols

Steps:

Step 1: Update cost.zod.ts

// packages/spec/src/ai/cost.zod.ts

/**

  • Token Usage Schema
  • Standardized across all AI operations
    */
    export const TokenUsageSchema = z.object({
    prompt: z.number().describe('Input tokens'),
    completion: z.number().describe('Output tokens'),
    total: z.number().describe('Total tokens'),
    });

export type TokenUsage = z.infer;

/**

  • AI Operation Cost Schema
  • Unified cost tracking for all AI operations
    */
    export const AIOperationCostSchema = z.object({
    operationId: z.string(),
    operationType: z.enum(['conversation', 'orchestration', 'prediction', 'rag', 'nlq']),
    agentName: z.string().optional().describe('Agent that performed the operation'),
    modelId: z.string(),
    tokens: TokenUsageSchema,
    cost: z.number().describe('Cost in USD'),
    timestamp: z.string().datetime(),
    metadata: z.record(z.any()).optional(),
    });

export type AIOperationCost = z.infer;

// Existing CostEntry extends this
export const CostEntrySchema = AIOperationCostSchema.extend({
object: z.string().optional(),
recordId: z.string().optional(),
});
Step 2: Update conversation.zod.ts

// packages/spec/src/ai/conversation.zod.ts

import { TokenUsageSchema, AIOperationCostSchema } from './cost.zod';

export const ConversationMessageSchema = z.object({
// ... existing fields
tokens: TokenUsageSchema.optional().describe('Token usage for this message'),
cost: z.number().optional().describe('Cost for this message in USD'),
});

export const ConversationSessionSchema = z.object({
// ... existing fields
totalTokens: TokenUsageSchema.optional(),
totalCost: z.number().optional(),
});
Step 3: Update orchestration.zod.ts

// packages/spec/src/ai/orchestration.zod.ts

import { TokenUsageSchema, AIOperationCostSchema } from './cost.zod';

export const AIOrchestrationExecutionResultSchema = z.object({
// ... existing fields
tokens: TokenUsageSchema.optional(),
cost: z.number().optional(),
});
Step 4: Add cost tracking to all AI schemas

// ai/agent.zod.ts, nlq.zod.ts, predictive.zod.ts, rag-pipeline.zod.ts
// Add tokens and cost fields to result schemas
Effort: 8 hours
Risk: Low
Impact: Unified cost tracking across AI stack

PR Title: feat(ai): unify cost tracking across all AI operations

Task 3.2: Unify Chart Types 📊

统一图表类型

Issue: Dashboard has 27 chart types, Report has 7

Steps:

Step 1: Create chart.zod.ts

// packages/spec/src/ui/chart.zod.ts (NEW)

import { z } from 'zod';

/**

  • Unified Chart Type Taxonomy
  • Shared by Dashboard and Report widgets.
    */
    export const ChartTypeSchema = z.enum([
    // Comparison
    'bar',
    'horizontal-bar',
    'column',
    'grouped-bar',
    'stacked-bar',

// Trend
'line',
'area',
'stacked-area',
'step-line',

// Distribution
'pie',
'donut',
'funnel',

// Relationship
'scatter',
'bubble',

// Composition
'treemap',
'sunburst',
'sankey',

// Performance
'gauge',
'metric',
'kpi',

// Geo
'choropleth',
'bubble-map',

// Advanced
'heatmap',
'radar',
'waterfall',
'box-plot',
'violin',

// Tabular
'table',
'pivot',
]);

export type ChartType = z.infer;

/**

  • Chart Configuration Base
    */
    export const ChartConfigSchema = z.object({
    type: ChartTypeSchema,
    title: z.string().optional(),
    description: z.string().optional(),
    showLegend: z.boolean().default(true),
    showDataLabels: z.boolean().default(false),
    colors: z.array(z.string()).optional(),
    });

export type ChartConfig = z.infer;
Step 2: Update dashboard.zod.ts

// packages/spec/src/ui/dashboard.zod.ts

import { ChartTypeSchema, ChartConfigSchema } from './chart.zod';

export const DashboardWidgetSchema = z.object({
// ... existing fields
chartConfig: ChartConfigSchema,
});
Step 3: Update report.zod.ts

// packages/spec/src/ui/report.zod.ts

import { ChartTypeSchema, ChartConfigSchema } from './chart.zod';

export const ReportChartSchema = ChartConfigSchema.extend({
// Report-specific extensions
xAxis: z.string(),
yAxis: z.string(),
groupBy: z.string().optional(),
});
Step 4: Update ui/index.ts

// packages/spec/src/ui/index.ts

export * from './chart.zod'; // NEW
// ... existing exports
Effort: 3 hours
Risk: Low
Impact: Consistent chart types

PR Title: refactor(ui): create unified chart type taxonomy


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

@vercel
Copy link
Copy Markdown

vercel bot commented Jan 30, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
spec Ready Ready Preview, Comment Jan 30, 2026 3:56pm

Request Review

Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
Copilot AI changed the title [WIP] Unify AI cost tracking across protocols feat(ai,ui): unify cost tracking and chart type taxonomy Jan 30, 2026
Copilot AI requested a review from hotlong January 30, 2026 16:03
@hotlong hotlong marked this pull request as ready for review January 30, 2026 16:13
Copilot AI review requested due to automatic review settings January 30, 2026 16:13
@github-actions github-actions bot added documentation Improvements or additions to documentation tests protocol:ui protocol:ai labels Jan 30, 2026
@github-actions
Copy link
Copy Markdown
Contributor

This PR is very large. Consider breaking it into smaller PRs for easier review.

@hotlong hotlong merged commit 856c549 into main Jan 30, 2026
11 of 13 checks passed
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR unifies cost tracking across AI operations and creates a standardized chart type taxonomy shared by Dashboard and Report widgets.

Changes:

  • Created standardized TokenUsageSchema and AIOperationCostSchema for unified AI cost tracking, integrating them into 6 AI protocols (conversation, orchestration, nlq, predictive, rag-pipeline, and cost itself)
  • Created new chart.zod.ts with 27 chart types organized by purpose (comparison, trend, distribution, relationship, composition, performance, geo, advanced, tabular), replacing fragmented chart type definitions in Dashboard (27 types) and Report (7 types)
  • Added 56 comprehensive tests (35 for cost tracking, 21 for chart taxonomy)

Reviewed changes

Copilot reviewed 39 out of 39 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
packages/spec/src/ai/cost.zod.ts Added TokenUsageSchema and AIOperationCostSchema for standardized cost tracking
packages/spec/src/ai/cost.test.ts Added 35 tests for token usage and AI operation cost schemas
packages/spec/src/ai/conversation.zod.ts Integrated unified TokenUsageSchema and added cost field to messages and sessions
packages/spec/src/ai/orchestration.zod.ts Added tokens and cost fields to execution results
packages/spec/src/ai/nlq.zod.ts Added cost tracking fields to NLQ response schema
packages/spec/src/ai/predictive.zod.ts Added cost tracking fields to prediction results
packages/spec/src/ai/rag-pipeline.zod.ts Added unified token/cost tracking with backward compatibility for deprecated tokensUsed
packages/spec/src/ui/chart.zod.ts New file defining unified chart type taxonomy with 27 chart types
packages/spec/src/ui/chart.test.ts Added 21 comprehensive tests for chart types and configurations
packages/spec/src/ui/dashboard.zod.ts Removed inline chart type enum, now imports from chart.zod.ts
packages/spec/src/ui/dashboard.test.ts Updated tests to use unified chart types, removed tests for obsolete types
packages/spec/src/ui/report.zod.ts Extended ChartConfigSchema for report-specific chart configuration
packages/spec/src/ui/report.test.ts Updated tests to validate new chart types
packages/spec/src/ui/index.ts Added export for chart schemas
packages/spec/json-schema/** Generated JSON schemas for new TypeScript schemas
content/docs/references/** Updated documentation to reflect new schemas and chart types

Comment on lines +165 to +167
/** Cost Tracking */
tokens: TokenUsageSchema.optional().describe('Token usage for this query'),
cost: z.number().nonnegative().optional().describe('Cost for this query in USD'),
Copy link

Copilot AI Jan 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new tokens and cost fields added to NLQResponseSchema lack test coverage. Consider adding tests to the nlq test file to validate these new fields, ensuring proper validation for the TokenUsageSchema structure and non-negative cost values.

Copilot uses AI. Check for mistakes.
Comment on lines +263 to +264
tokens: TokenUsageSchema.optional().describe('Token usage for this prediction (if AI-powered)'),
cost: z.number().nonnegative().optional().describe('Cost for this prediction in USD'),
Copy link

Copilot AI Jan 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new tokens and cost fields added to PredictionResultSchema lack test coverage. Consider adding tests to the predictive test file to validate these new fields, especially for the case where predictions are AI-powered and include token usage and cost information.

Copilot uses AI. Check for mistakes.
Comment on lines +255 to +257
tokens: TokenUsageSchema.optional().describe('Token usage for this query'),
cost: z.number().nonnegative().optional().describe('Cost for this query in USD'),
tokensUsed: z.number().int().optional().describe('Deprecated: use tokens.total instead'),
Copy link

Copilot AI Jan 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new tokens and cost fields added to RAGQueryResponseSchema lack test coverage. Consider adding tests to the rag-pipeline test file to validate these new fields. Also note that the deprecated tokensUsed field should have tests to ensure backward compatibility while guiding users to migrate to the new tokens.total field.

Copilot uses AI. Check for mistakes.
Comment on lines +85 to +86
tokens: TokenUsageSchema.optional().describe('Token usage for this message'),
cost: z.number().nonnegative().optional().describe('Cost for this message in USD'),
Copy link

Copilot AI Jan 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new cost field added to ConversationMessageSchema lacks test coverage. The existing test file conversation.test.ts contains tests for the tokens field but does not validate the new cost field. Consider adding tests to ensure the cost field validates correctly with positive numbers, rejects negative values, and works properly when optional.

Copilot uses AI. Check for mistakes.
Comment on lines +197 to +198
totalTokens: TokenUsageSchema.optional().describe('Total tokens across all messages'),
totalCost: z.number().nonnegative().optional().describe('Total cost for this session in USD'),
Copy link

Copilot AI Jan 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new totalTokens and totalCost fields added to ConversationSessionSchema lack test coverage. The existing test file conversation.test.ts does not include validation tests for these new fields. Consider adding tests to ensure these fields work correctly, especially for validating that totalCost only accepts non-negative numbers and totalTokens follows the TokenUsageSchema structure.

Copilot uses AI. Check for mistakes.
Comment on lines +223 to +224
tokens: TokenUsageSchema.optional().describe('Total token usage for this execution'),
cost: z.number().nonnegative().optional().describe('Total cost for this execution in USD'),
Copy link

Copilot AI Jan 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new tokens and cost fields added to AIOrchestrationExecutionResultSchema lack test coverage. Consider adding tests to the orchestration test file to validate these new fields, including checking that tokens follows the TokenUsageSchema structure and cost only accepts non-negative numbers.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation protocol:ai protocol:ui size/xl tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants