Python: feat(Azure AI Search Context Provider): Add Azure AI Search Context Provider with toggle for "semantic" mode and "agentic" mode in azure ai by farzad528 · Pull Request #1546 · microsoft/agent-framework

farzad528 · 2025-10-17T16:45:51Z

Motivation and Context

This PR adds a first-class Azure AI Search context provider to the core framework, enabling agents to ground responses using both:

Semantic search over existing Azure AI Search indexes, and
Agentic retrieval via the new Knowledge Base APIs (introduced in azure-search-documents==11.7.0b2).

The goal is to support knowledge-grounded reasoning across enterprise content, using either lightweight semantic retrieval or multi-hop, citation-capable agentic grounding. This also centralizes Search support in the core package, making it available to all agents without extra Azure-specific packages.

Description

Key changes:

Added AzureAISearchContextProvider in core/agent_framework/azure/_search_provider.py and exported via: from agent_framework.azure import AzureAISearchContextProvider

Upgraded to azure-search-documents==11.7.0b2 and fully migrated from deprecated KnowledgeAgent types to the new KnowledgeBase API:
- KnowledgeAgent → KnowledgeBase
- KnowledgeAgentMessage → KnowledgeBaseMessage
- Module: azure.search.documents.agent → azure.search.documents.knowledgebases
Added dual authentication support for Knowledge Bases:
- API key
- Managed identity
Added azure_ai_with_search_context.py sample demonstrating:
- Semantic mode (hybrid search + semantic rerank)
- Agentic mode (Knowledge Base retrieval using full message history)
Updated README.md with correct sample references and usage guidance.

Features:

Semantic mode
- Hybrid (vector + keyword) search
- Semantic ranking
- Internal fan-out: k_nearest_neighbors=50 → reranked to top_k (default 5)
- Uses captions by default for concise grounding
Agentic mode
- Multi-hop retrieval via Knowledge Base APIs
- Supports citations (not yet surfaced in Context; planned follow-up)
- Accepts full message list for conversational grounding
Implementation improvements
- Caches KnowledgeBaseRetrievalClient in __aenter__
- Handles streaming + non-streaming modes
- Async context manager support
- Vectorizer auto-discovery, system-message filtering, and configurable limits tracked for follow-ups

Contribution Checklist

Code builds cleanly with no errors or warnings
PR follows the Contribution Guidelines
Unit tests pass (21/21) + core test suite (1123 passed)
Pre-commit checks pass (ruff, pyright, mypy)
Sample and README updated
[] Breaking change:

Copilot

Pull Request Overview

This PR adds a Python Azure AI Search Context Provider with support for two retrieval modes: semantic (fast hybrid search) and agentic (slower multi-hop reasoning with Knowledge Bases).

Implements AzureAISearchContextProvider with dual-mode functionality for different use cases
Adds comprehensive sample demonstrating both semantic and agentic retrieval modes
Integrates azure-search-documents dependency for search capabilities

Reviewed Changes

Copilot reviewed 5 out of 6 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
`python/packages/azure-ai/agent_framework_azure_ai/_search_provider.py`	Core implementation of AzureAISearchContextProvider with semantic and agentic modes
`python/samples/getting_started/agents/azure_ai/azure_ai_with_search_context.py`	Sample demonstrating usage of the search context provider with both modes
`python/packages/azure-ai/pyproject.toml`	Adds azure-search-documents dependency
`python/packages/azure-ai/agent_framework_azure_ai/__init__.py`	Exports the new AzureAISearchContextProvider class
`python/samples/getting_started/agents/azure_ai/README.md`	Documents the new sample and required environment variables

python/packages/azure-ai/agent_framework_azure_ai/_search_provider.py

markwallace-microsoft · 2025-10-17T16:47:32Z

Python Test Coverage Report •

File	Stmts	Miss	Cover	Missing
packages/core/agent_framework/azure
_search_provider.py	215	78	63%	59–60, 227, 281–282, 319–320, 324, 339, 358, 361–362, 365–366, 368, 372, 374, 376, 395, 401, 403, 405, 407–411, 414, 417, 419–423, 425, 431, 433, 437, 439–441, 443–444, 446, 450, 453, 455, 461–462, 498, 528–530, 557, 561, 563, 565, 639, 642, 651, 654–655, 658, 661, 663–664, 666–667, 669–670, 672–673, 676, 697–701
TOTAL	15203	2285	84%

Python Unit Test Overview

Tests	Skipped	Failures	Errors	Time
2060	127 💤	0 ❌	0 🔥	42.186s ⏱️

python/samples/getting_started/agents/azure_ai/README.md

moonbox3 · 2025-10-24T01:12:13Z

@farzad528 there are also several pyright typing issues we'll need to address: https://github.com/microsoft/agent-framework/actions/runs/18756209717/job/53508713618?pr=1546#step:5:269

You'll want to make sure you have the pre-commit checks install so they run before committing code: https://github.com/microsoft/agent-framework/blob/main/python/DEV_SETUP.md#after-installing-uv

dmytrostruk

Looks good in general, just need to resolve CI errors and it would be great to add some unit tests.

python/packages/core/agent_framework/azure/_search_provider.py

moonbox3 · 2025-10-28T00:10:25Z

@farzad528 there are also several pyright typing issues we'll need to address: https://github.com/microsoft/agent-framework/actions/runs/18756209717/job/53508713618?pr=1546#step:5:269

You'll want to make sure you have the pre-commit checks install so they run before committing code: https://github.com/microsoft/agent-framework/blob/main/python/DEV_SETUP.md#after-installing-uv

This still needs to be handled, as well. Pre-commit checks are failing.

python/packages/core/pyproject.toml

python/packages/core/agent_framework/azure/_search_provider.py

python/samples/getting_started/agents/azure_ai/README.md

python/packages/core/agent_framework/azure/_search_provider.py

…t#1546)

- Fixes mypy error: Cannot assign to final name 'DEFAULT_CONTEXT_PROMPT' - Base class ContextProvider already defines this as Final - Instance variable self.context_prompt correctly set in __init__ (Line 197) Related to microsoft#1546

…t#1546)

- Fixes mypy error: Cannot assign to final name 'DEFAULT_CONTEXT_PROMPT' - Base class ContextProvider already defines this as Final - Instance variable self.context_prompt correctly set in __init__ (Line 197) Related to microsoft#1546

…ch-agentic-rag

- Add type annotation to DEFAULT_CONTEXT_PROMPT - Add type annotation to vectorizable_fields - Add union type annotation to vector_queries

- Rename DEFAULT_CONTEXT_PROMPT to _DEFAULT_SEARCH_CONTEXT_PROMPT to avoid conflict with base class Final variable - Update test to use new constant name - All core package tests passing (1123 passed)

…rag' into feature/azure-ai-search-agentic-rag

eavanvalkenburg · 2025-11-18T08:57:31Z

python/packages/core/pyproject.toml

    "openai>=1.99.0",
    "azure-identity>=1,<2",
    "mcp[ws]>=1.13",
+    "azure-search-documents==11.7.0b2",


given this dependency (beta status) all this code needs to be moved into a seperate package, while still importing through the .azure module. So we need to create a new package called agent-framework-azureaisearch or agent-framework-aisearch or something similar, that exports the same files, but uses lazy loading in the core/agent_framework/azure/__init__.py file.

eavanvalkenburg · 2025-11-18T08:59:11Z

python/packages/core/agent_framework/azure/_search_provider.py

+        self,
+        endpoint: str,
+        index_name: str,
+        credential: AzureKeyCredential | AsyncTokenCredential,


let's also allow folks to pass in a key directly, and then we create the AzureKeyCredential in code.

eavanvalkenburg · 2025-11-18T09:00:07Z

python/packages/core/agent_framework/azure/_search_provider.py

+
+    def __init__(
+        self,
+        endpoint: str,


I would like a settings class to be used for this, index_name and the key, and potentially also for azure_ai_project_endpoint, model, or azure openai config.

eavanvalkenburg · 2025-11-18T09:02:46Z

python/packages/core/agent_framework/azure/_search_provider.py

+        top_k: int = 5,
+        semantic_configuration_name: str | None = None,
+        vector_field_name: str | None = None,
+        embedding_function: Any | None = None,


this should also be typed, otherwise I have no clue what to pass in here!

eavanvalkenburg · 2025-11-18T09:04:10Z

python/packages/core/agent_framework/azure/_search_provider.py

+        """
+        # Convert to list and filter to USER/ASSISTANT messages with text only
+        messages_list = [messages] if isinstance(messages, ChatMessage) else list(messages)
+        from agent_framework import Role


this can be a top-level import

…ge) (#2328) * Python: Fix pyright errors and move search provider to core (#1546) * address pablo coments * update azure ai search pypi version to latest prev * init update * Fix MyPy type annotation errors in search provider - Add type annotation to DEFAULT_CONTEXT_PROMPT - Add type annotation to vectorizable_fields - Add union type annotation to vector_queries * Fix DEFAULT_CONTEXT_PROMPT MyPy error and update test - Rename DEFAULT_CONTEXT_PROMPT to _DEFAULT_SEARCH_CONTEXT_PROMPT to avoid conflict with base class Final variable - Update test to use new constant name - All core package tests passing (1123 passed) * Python: Move Azure AI Search to separate package per PR feedback Addresses reviewer feedback from PR #1546 by isolating the beta dependency (azure-search-documents==11.7.0b2) into a new agent-framework-aisearch package. Changes: - Created new agent-framework-aisearch package with complete structure - Moved AzureAISearchContextProvider from core to aisearch package - Added AzureAISearchSettings class for environment variable auto-loading - Added support for direct API key string (auto-converts to AzureKeyCredential) - Added azure_openai_api_key parameter for Knowledge Base authentication - Updated embedding_function type to Callable[[str], Awaitable[list[float]]] - Moved Role import to top-level imports - Maintained lazy loading through agent_framework.azure module - Removed beta dependency from core package - Updated all tests to use new package location - All quality checks pass: ruff format/lint, pyright, mypy (0 errors) - All 21 unit tests pass with 59% coverage Semantic search mode verified working with both API key and managed identity authentication. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Python: Clarify top_k parameter only applies to semantic mode Updated documentation to clarify that the top_k parameter only affects semantic search mode. In agentic mode, the server-side Knowledge Base determines retrieval based on query complexity and reasoning effort. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Python: Add Knowledge Base output mode and retrieval reasoning effort parameters Added support for configurable Knowledge Base behavior in agentic mode: - knowledge_base_output_mode: "extractive_data" (default) or "answer_synthesis" Some knowledge sources require answer_synthesis mode for proper functionality. - retrieval_reasoning_effort: "minimal" (default), "medium", or "low" Controls query planning complexity and multi-hop reasoning depth. These parameters give users fine-grained control over Knowledge Base behavior and enable support for knowledge sources that require answer synthesis. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * effort and outputmode query params * Address PR review feedback for Azure AI Search context provider * comments eduward * ed latest comments --------- Co-authored-by: Farzad Sunavala <farzad.sunavala.enovate.ai> Co-authored-by: farzad528 <farzad528@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com>

…ge) (microsoft#2328) * Python: Fix pyright errors and move search provider to core (microsoft#1546) * address pablo coments * update azure ai search pypi version to latest prev * init update * Fix MyPy type annotation errors in search provider - Add type annotation to DEFAULT_CONTEXT_PROMPT - Add type annotation to vectorizable_fields - Add union type annotation to vector_queries * Fix DEFAULT_CONTEXT_PROMPT MyPy error and update test - Rename DEFAULT_CONTEXT_PROMPT to _DEFAULT_SEARCH_CONTEXT_PROMPT to avoid conflict with base class Final variable - Update test to use new constant name - All core package tests passing (1123 passed) * Python: Move Azure AI Search to separate package per PR feedback Addresses reviewer feedback from PR microsoft#1546 by isolating the beta dependency (azure-search-documents==11.7.0b2) into a new agent-framework-aisearch package. Changes: - Created new agent-framework-aisearch package with complete structure - Moved AzureAISearchContextProvider from core to aisearch package - Added AzureAISearchSettings class for environment variable auto-loading - Added support for direct API key string (auto-converts to AzureKeyCredential) - Added azure_openai_api_key parameter for Knowledge Base authentication - Updated embedding_function type to Callable[[str], Awaitable[list[float]]] - Moved Role import to top-level imports - Maintained lazy loading through agent_framework.azure module - Removed beta dependency from core package - Updated all tests to use new package location - All quality checks pass: ruff format/lint, pyright, mypy (0 errors) - All 21 unit tests pass with 59% coverage Semantic search mode verified working with both API key and managed identity authentication. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Python: Clarify top_k parameter only applies to semantic mode Updated documentation to clarify that the top_k parameter only affects semantic search mode. In agentic mode, the server-side Knowledge Base determines retrieval based on query complexity and reasoning effort. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Python: Add Knowledge Base output mode and retrieval reasoning effort parameters Added support for configurable Knowledge Base behavior in agentic mode: - knowledge_base_output_mode: "extractive_data" (default) or "answer_synthesis" Some knowledge sources require answer_synthesis mode for proper functionality. - retrieval_reasoning_effort: "minimal" (default), "medium", or "low" Controls query planning complexity and multi-hop reasoning depth. These parameters give users fine-grained control over Knowledge Base behavior and enable support for knowledge sources that require answer synthesis. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * effort and outputmode query params * Address PR review feedback for Azure AI Search context provider * comments eduward * ed latest comments --------- Co-authored-by: Farzad Sunavala <farzad.sunavala.enovate.ai> Co-authored-by: farzad528 <farzad528@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com>

Copilot AI review requested due to automatic review settings October 17, 2025 16:45

markwallace-microsoft added documentation Improvements or additions to documentation python labels Oct 17, 2025

Copilot AI reviewed Oct 17, 2025

View reviewed changes

moonbox3 reviewed Oct 24, 2025

View reviewed changes

python/samples/getting_started/agents/azure_ai/README.md Outdated Show resolved Hide resolved

dmytrostruk reviewed Oct 24, 2025

View reviewed changes

moonbox3 reviewed Oct 24, 2025

View reviewed changes

python/packages/core/agent_framework/azure/_search_provider.py Outdated Show resolved Hide resolved

farzad528 requested a review from moonbox3 October 27, 2025 22:35

eavanvalkenburg reviewed Oct 29, 2025

View reviewed changes

pablocastro reviewed Oct 31, 2025

View reviewed changes

farzad528 closed this Nov 2, 2025

farzad528 force-pushed the feature/azure-ai-search-agentic-rag branch from 1d770e3 to a9a1b1e Compare November 2, 2025 11:08

Python: Fix pyright errors and move search provider to core (microsof…

ed8af31

…t#1546)

farzad528 reopened this Nov 2, 2025

address pablo coments

8de2673

This was referenced Nov 15, 2025

Python: Fix CI issues and add comprehensive tests for Azure AI Search Context Provider #2246

Closed

Python: Alternative fix for #1546 - Add comprehensive tests and fix CI issues for Azure AI Search Context Provider #2247

Closed

hayato-kishikawa pushed a commit to hayato-kishikawa/agent-framework that referenced this pull request Nov 15, 2025

Python: Fix pyright errors and move search provider to core (microsof…

dc7b011

…t#1546)

farzad528 closed this Nov 16, 2025

farzad528 force-pushed the feature/azure-ai-search-agentic-rag branch from 8de2673 to 45dba6b Compare November 16, 2025 14:43

github-merge-queue bot temporarily deployed to integration November 17, 2025 00:10 Inactive

eavanvalkenburg temporarily deployed to integration November 17, 2025 00:12 — with GitHub Actions Inactive

farzad528 and others added 4 commits November 17, 2025 16:31

update azure ai search pypi version to latest prev

0cd48ab

Merge remote-tracking branch 'origin/main' into feature/azure-ai-sear…

88886a3

…ch-agentic-rag

init update

a9244c2

Merge branch 'microsoft:main' into feature/azure-ai-search-agentic-rag

5aa6c32

farzad528 reopened this Nov 17, 2025

farzad528 and others added 7 commits November 17, 2025 19:08

Merge branch 'main' into feature/azure-ai-search-agentic-rag

c0d2e51

Fix MyPy type annotation errors in search provider

c7d1e79

- Add type annotation to DEFAULT_CONTEXT_PROMPT - Add type annotation to vectorizable_fields - Add union type annotation to vector_queries

Merge branch 'main' into feature/azure-ai-search-agentic-rag

69e5841

Fix DEFAULT_CONTEXT_PROMPT MyPy error and update test

df2aaac

- Rename DEFAULT_CONTEXT_PROMPT to _DEFAULT_SEARCH_CONTEXT_PROMPT to avoid conflict with base class Final variable - Update test to use new constant name - All core package tests passing (1123 passed)

Merge remote-tracking branch 'origin/feature/azure-ai-search-agentic-…

f9e6cab

…rag' into feature/azure-ai-search-agentic-rag

Merge branch 'main' into feature/azure-ai-search-agentic-rag

8043512

Merge branch 'main' into feature/azure-ai-search-agentic-rag

8923716

eavanvalkenburg reviewed Nov 18, 2025

View reviewed changes

Merge branch 'microsoft:main' into feature/azure-ai-search-agentic-rag

316b39e

farzad528 mentioned this pull request Nov 19, 2025

Python: Feature/azure ai search agentic rag (search as separate package) #2328

Merged

3 tasks

eavanvalkenburg closed this Nov 21, 2025

Conversation

farzad528 commented Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation and Context

Description

Contribution Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

markwallace-microsoft commented Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Python Unit Test Overview

Uh oh!

Uh oh!

moonbox3 commented Oct 24, 2025

Uh oh!

dmytrostruk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

moonbox3 commented Oct 28, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

eavanvalkenburg Nov 18, 2025

Choose a reason for hiding this comment

Uh oh!

eavanvalkenburg Nov 18, 2025

Choose a reason for hiding this comment

Uh oh!

eavanvalkenburg Nov 18, 2025

Choose a reason for hiding this comment

Uh oh!

eavanvalkenburg Nov 18, 2025

Choose a reason for hiding this comment

Uh oh!

eavanvalkenburg Nov 18, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

farzad528 commented Oct 17, 2025 •

edited

Loading

markwallace-microsoft commented Oct 17, 2025 •

edited

Loading