Adding semantic caching with Azure Managed Redis #3024
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Add SemanticCacheOptions with similarity threshold, max results, TTL - Add AzureManagedRedisOptions for Redis connection configuration - Add EmbeddingProviderOptions for Azure OpenAI configuration - Wire semantic cache options into RuntimeOptions and RuntimeConfig - Add UserProvided flags following DAB repository patterns
- Add SemanticCacheOptionsConverterFactory with validation - Add AzureManagedRedisOptionsConverterFactory - Add EmbeddingProviderOptionsConverterFactory - Register converters in RuntimeConfigLoader.GetSerializationOptions() - Validate similarity threshold (0.0-1.0) and numeric fields
… Redis - Implement AzureOpenAIEmbeddingService with exponential backoff retry - Implement RedisVectorStore with RediSearch vector similarity (KNN) - Implement SemanticCacheService orchestration layer - Add SemanticCacheResult DTO - Register services in DI with conditional configuration validation - Use COSINE distance metric for text embeddings - Support automatic Redis vector index creation
- Architecture overview and component descriptions - Configuration examples and parameter reference - Usage patterns and integration examples - Performance characteristics and scalability guidance - Troubleshooting guide and monitoring recommendations
Add ValidateSemanticCacheConfiguration() method to RuntimeConfigValidator to ensure semantic cache is properly configured when enabled. **Validations:** - Validates Azure Managed Redis connection string is not null/empty - Validates embedding provider endpoint, API key, and model are configured - Validates similarity-threshold is between 0.0 and 1.0 - Validates max-results and expire-seconds are positive integers - Integrated into ValidateConfigProperties() for startup validation Completes semantic caching infrastructure implementation.
…r semantic caching
…rFactory, updating integration tests, e2e tests and readme file
Contributor
|
Hi @robertopc1, once you think the PR is ready for review please change it from a draft to an open PR. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why make this change?
Closes #3023 — Adds semantic caching support so repeated (or semantically equivalent) SQL queries can be served from cache instead of re-executing against the database, reducing latency and database load. This also enables “near-duplicate” query reuse by caching against vector similarity rather than exact string matching.
Additional discussion/setup notes: semantic-cache-real-azure-openai-setup.md
What is this change?
Introduces a new semantic caching pipeline for SQL query execution (MSSQL/MySQL/PostgreSQL) backed by:
Embeddings generated via an IEmbeddingService implementation (Azure OpenAI).
Vector storage + similarity search via ISemanticCache implemented on top of Azure Managed Redis vector capabilities.
Adds runtime config support for semantic caching:
New config object models (SemanticCacheOptions, EmbeddingProviderOptions, AzureManagedRedisOptions) and JSON converter factories.
Runtime config loading + validation updates to enforce required semantic cache configuration.
Wires semantic cache through the execution stack:
Updates QueryManagerFactory / QueryEngineFactory and SQL executors to use the semantic cache-aware QueryExecutor flow.
Updates service startup to register semantic cache services.
Adds CLI support to generate semantic cache configuration via config generation paths (CLI options + config generator updates).
References:
Real Azure setup + testing guide: semantic-cache-real-azure-openai-setup.md
Redis vector similarity search concepts (for reviewers): https://redis.io/docs/latest/develop/interact/search-and-query/query/vector-search/
How was this tested?
Integration Tests — SemanticCacheIntegrationTests.cs
Unit Tests — SemanticCacheOptionsTests.cs, SemanticCacheServiceTests.cs, AzureOpenAIEmbeddingServiceTests.cs
E2E Tests — SemanticCacheE2ETests.cs and CLI e2e updates in EndToEndTests.cs
Sample Request(s)
# First request (cache miss -> DB execution + cache write) curl -s "http://localhost:5000/api/Books?$filter=title eq 'Dune'"# Second request (expected semantic cache hit -> served from cache) curl -s "http://localhost:5000/api/Books?$filter=title eq 'Dune'"# First request (cache miss) query { books(filter: { title: { eq: "Dune" } }) { id title } }# Re-run the same (or semantically equivalent) query (expected cache hit) query { books(filter: { title: { eq: "Dune" } }) { id title } }