Skip to content

feat: add token savings estimation to tool responses#61

Open
halindrome wants to merge 7 commits intoDeusData:mainfrom
halindrome:feat/token-savings-estimation
Open

feat: add token savings estimation to tool responses#61
halindrome wants to merge 7 commits intoDeusData:mainfrom
halindrome:feat/token-savings-estimation

Conversation

@halindrome
Copy link

Summary

Adds _meta token savings estimation metadata to MCP tool responses, inspired by jcodemunch. This gives users visibility into how many tokens (and dollars) are saved by using structured graph queries instead of reading entire source files.

  • New internal/metrics packageTokenMetadata struct, EstimateTokens (char/4 heuristic), CalculateSavings, and Tracker with atomic JSON persistence to ~/.cache/codebase-memory-mcp/savings.json
  • Config keysmetrics_enabled (default: true), pricing_model (claude-sonnet/claude-opus/gpt-4o/custom), custom_price_per_token, plus GetFloat64 helper on ConfigStore
  • get_code_snippet instrumented — baseline = full source file size via os.Stat (the file the user would have read)
  • search_graph instrumented — baseline = sum of unique source file sizes in results (all files the user would scan to find those symbols)
  • Fail-open — if metrics are disabled or os.Stat fails, responses fall back to standard jsonResult with no _meta (no regression)
  • No new external dependencies — uses character-based heuristic only

Example _meta in response

{
  "_meta": {
    "tokens_saved": 900,
    "baseline_tokens": 1000,
    "response_tokens": 100,
    "cost_avoided": 0.0135,
    "compression_ratio": 0.1
  }
}

Test plan

  • go test ./internal/metrics/... -count=1 — 8 tests (calculations, tracker persistence, load existing, missing dir)
  • go test ./internal/tools/... -count=1 — includes TestGetCodeSnippet_MetaField and TestSearchGraph_MetaField
  • go test ./internal/store/... -count=1 — config constants and GetFloat64
  • golangci-lint run ./... — 0 new issues
  • No new external dependencies added

🤖 Generated with Claude Code

shanemccarron-maker and others added 7 commits March 15, 2026 12:05
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Extract metrics block into searchResultWithMeta to reduce cognitive
complexity. Add TestGetCodeSnippet_MetaField to snippet_test.go.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Adds TestSearchGraph_MetaField in internal/tools/search_test.go
- Verifies _meta field is present in search_graph response when metricsTracker is attached
- Asserts tokens_saved >= 0, baseline_tokens >= 0 (may be 0 for test fixtures), response_tokens > 0

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants