Add MCP optimizer implementation for semantic tool discovery

[therealnb]

Add MCP Optimizer Implementation for Semantic Tool Discovery

This PR adds the complete MCP optimizer implementation to vMCP, enabling semantic tool discovery and reducing token usage for LLMs working with large toolsets.

Overview

The optimizer allows vMCP to expose optim.find_tool and optim.call_tool operations instead of all backend tools directly. This reduces token usage by allowing LLMs to discover relevant tools on demand via semantic search rather than receiving all tool definitions upfront.

Features

Core Optimizer Package

Semantic Tool Search (pkg/optimizer/)

• Vector embeddings (384-dim) for semantic similarity search
• Full-text search via SQLite FTS5 for BM25 text matching
• Hybrid search combining semantic and BM25 results (configurable ratio)
• Multiple embedding backends:
  • Ollama (local HTTP API)
  • OpenAI-compatible (vLLM, OpenAI, etc.)
  • Placeholder (deterministic hash-based, for testing)

Token Counting (pkg/optimizer/tokens/)

• LLM cost estimation based on token counts
• Supports monitoring token usage and optimization effectiveness

Database Layer (pkg/optimizer/db/)

• SQLite-based storage with sqlite-vec for vector similarity search
• FTS5 for full-text search
• Hybrid search implementation combining both approaches
• Persistent and in-memory storage options

Ingestion Service (pkg/optimizer/ingestion/)

• Ingests tools from all backends in the group
• Generates embeddings for tool metadata
• Maintains searchable index of all available tools

vMCP Integration

Optimizer Endpoints (pkg/vmcp/optimizer/)

• optim.find_tool: Semantic and string-based tool discovery
• optim.call_tool: Tool invocation with automatic routing
• Integration with vMCP server lifecycle
• Comprehensive test coverage (unit, integration, semantic search, string matching)

Server Integration (pkg/vmcp/server/)

• Optimizer initialization and lifecycle management
• Session-based optimizer instances
• Integration with discovery manager and backend registry

Router Updates (pkg/vmcp/router/)

• Special handling for optim_* prefixed tools
• Prevents routing optimizer tools to backends (handled by vMCP itself)

Kubernetes Operator Support

Service Resolution (cmd/thv-operator/pkg/vmcpconfig/converter.go)

• Resolves Kubernetes Service names to URLs for embedding services
• Handles embeddingService → embeddingURL conversion
• Supports in-cluster deployments

CRD Schema (deploy/charts/operator-crds/)

• Complete optimizer configuration schema
• Supports all optimizer features (embeddings, persistence, hybrid search)
• Documentation updates

Configuration

OptimizerConfig (pkg/vmcp/config/config.go)

• Comprehensive configuration options:
  • enabled: Enable/disable optimizer
  • embeddingBackend: Choose embedding provider
  • embeddingURL: Embedding service URL
  • embeddingModel: Model name for embeddings
  • embeddingDimension: Vector dimension
  • persistPath: Optional persistence path
  • ftsDBPath: FTS5 database path
  • hybridSearchRatio: Semantic vs BM25 mix (0-100%)
  • embeddingService: Kubernetes service name (K8s only)

CLI Integration (cmd/vmcp/app/commands.go)

• Optimizer configuration parsing from YAML
• Runtime configuration and initialization
• Logging and status reporting

Build System

Build Tags (Taskfile.yml)

• Added -tags="fts5" build flag for SQLite FTS5 support
• Required for optimizer functionality
• Applied to all vmcp builds (build, install)

Test Task (Taskfile.yml)

• Added test-optimizer task for optimizer integration tests
• Uses sqlite-vec for vector search testing

Examples & Scripts

Example Configuration (examples/vmcp-config-optimizer.yaml)

• Complete example showing optimizer configuration
• Demonstrates all configuration options

Helper Scripts (scripts/)

• test-optimizer-with-sqlite-vec.sh: Integration testing
• inspect-optimizer-db.sh: Database inspection
• query-optimizer-db.sh: Query testing
• Various chromem inspection tools

Documentation

• Optimizer package documentation in pkg/optimizer/README.md
• Integration guide in pkg/optimizer/INTEGRATION.md
• CRD API documentation updates
• Example configurations

Testing

• Comprehensive unit tests for all optimizer components
• Integration tests for optimizer endpoints
• Semantic search test suite
• String matching test suite
• E2E tests for Kubernetes deployments

Dependencies

• chromem-go: Vector database for embeddings
• sqlite-vec: SQLite extension for vector similarity search
• go.uber.org/mock: Mock generation for tests

Build Requirements

• Requires -tags="fts5" build flag for FTS5 support
• SQLite with FTS5 extension
• sqlite-vec extension for vector search

Related

• Depends on infrastructure improvements from optimizer-enablers PR
• Split from Merge jerm/2026-01-13-optimizer-in-vmcp into main #3373
• Implements semantic tool discovery as described in optimizer documentation

Large PR Justification

• This is the second part of a two part PR.

[github-actions]

Large PR Detected

This PR exceeds 1000 lines of changes and requires justification before it can be reviewed.

How to unblock this PR:

Add a section to your PR description with the following format:

## Large PR Justification

[Explain why this PR must be large, such as:]
- Generated code that cannot be split
- Large refactoring that must be atomic
- Multiple related changes that would break if separated
- Migration or data transformation

Alternative:

Consider splitting this PR into smaller, focused changes (< 1000 lines each) for easier review and reduced risk.

See our Contributing Guidelines for more details.

---

This review will be automatically dismissed once you add the justification section.

Large PR justification has been provided. Thank you!

[github-actions]

✅ Large PR justification has been provided. The size review has been dismissed and this PR can now proceed with normal review.

[codecov]

Codecov Report

❌ Patch coverage is 45.54580% with 868 lines in your changes missing coverage. Please review.
✅ Project coverage is 64.49%. Comparing base (e00d514) to head (21f90c6).

Files with missing lines	Patch %	Lines
pkg/vmcp/optimizer/optimizer.go	10.07%	352 Missing and 5 partials ⚠️
pkg/vmcp/optimizer/internal/ingestion/service.go	0.00%	157 Missing ⚠️
pkg/vmcp/optimizer/internal/db/fts.go	69.30%	48 Missing and 14 partials ⚠️
pkg/vmcp/optimizer/internal/embeddings/ollama.go	26.22%	42 Missing and 3 partials ⚠️
pkg/vmcp/optimizer/internal/embeddings/manager.go	47.56%	32 Missing and 11 partials ⚠️
pkg/vmcp/optimizer/internal/db/backend_tool.go	63.30%	20 Missing and 20 partials ⚠️
pkg/vmcp/server/server.go	10.00%	23 Missing and 4 partials ⚠️
pkg/vmcp/optimizer/internal/db/hybrid.go	64.86%	17 Missing and 9 partials ⚠️
pkg/vmcp/optimizer/internal/db/db.go	75.58%	17 Missing and 4 partials ⚠️
cmd/vmcp/app/commands.go	0.00%	18 Missing ⚠️
... and 10 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3440      +/-   ##
==========================================
- Coverage   65.15%   64.49%   -0.67%     
==========================================
  Files         398      412      +14     
  Lines       38821    40301    +1480     
==========================================
+ Hits        25295    25992     +697     
- Misses      11564    12266     +702     
- Partials     1962     2043      +81     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.