Skip to content

Python: Phase 2: Embedding clients for Ollama, Bedrock, and Azure AI Inference#4207

Merged
eavanvalkenburg merged 4 commits intomicrosoft:mainfrom
eavanvalkenburg:feature/embedding-phase2
Feb 25, 2026
Merged

Python: Phase 2: Embedding clients for Ollama, Bedrock, and Azure AI Inference#4207
eavanvalkenburg merged 4 commits intomicrosoft:mainfrom
eavanvalkenburg:feature/embedding-phase2

Conversation

@eavanvalkenburg
Copy link
Member

Summary

Add embedding client implementations to existing provider packages as part of the vector store & embeddings port.

New embedding clients

  • OllamaEmbeddingClient — Text embeddings via Ollama's embed API (packages/ollama/)
  • BedrockEmbeddingClient — Text embeddings via Amazon Titan on Bedrock (packages/bedrock/)
  • AzureAIInferenceEmbeddingClient — Text and image embeddings via Azure AI Inference (packages/azure-ai/)
    • Accepts Content | str input — dispatches text to EmbeddingsClient and images to ImageEmbeddingsClient
    • Separate model IDs for text (AZURE_AI_INFERENCE_EMBEDDING_MODEL_ID) and image (AZURE_AI_INFERENCE_IMAGE_EMBEDDING_MODEL_ID)
    • Async context manager support for proper resource cleanup

Additional changes

  • Rename EmbeddingCoTEmbeddingT, EmbeddingOptionsCoTEmbeddingOptionsT in core
  • Add otel_provider_name passthrough to all embedding clients (OpenAI, Azure OpenAI, Ollama, Bedrock, Azure AI Inference)
  • Register integration pytest marker in all packages
  • Add lazy-loading namespace exports for Ollama and Bedrock embeddings in core
  • Add azure-ai-inference dependency to azure-ai package
  • Add image embedding sample using Cohere-embed-v3-english

Testing

  • Unit tests for all three new clients (Ollama: 7, Bedrock: 7, Azure AI Inference: 17)
  • Integration tests with three-marker pattern (@flaky, @integration, @skip_if_...)

Fixes #4164
Part of #1188

Copilot AI review requested due to automatic review settings February 24, 2026 11:26
@eavanvalkenburg eavanvalkenburg requested a review from a team as a code owner February 24, 2026 11:26
@markwallace-microsoft markwallace-microsoft added documentation Improvements or additions to documentation python labels Feb 24, 2026
@github-actions github-actions bot changed the title Phase 2: Embedding clients for Ollama, Bedrock, and Azure AI Inference Python: Phase 2: Embedding clients for Ollama, Bedrock, and Azure AI Inference Feb 24, 2026
@markwallace-microsoft
Copy link
Member

markwallace-microsoft commented Feb 24, 2026

Python Test Coverage

Python Test Coverage Report •
FileStmtsMissCoverMissing
packages/azure-ai/agent_framework_azure_ai
   _embedding_client.py1131289%152, 167–170, 174, 178, 228, 238, 249, 258, 284
packages/core/agent_framework
   _clients.py85396%298, 491, 493
   _types.py10248691%59, 68–69, 123, 128, 147, 149, 153, 157, 159, 161, 163, 181, 185, 211, 233, 238, 243, 247, 273, 277, 632–633, 1004, 1066, 1083, 1101, 1106, 1124, 1134, 1151–1152, 1154, 1172–1173, 1175, 1182–1183, 1185, 1220, 1231–1232, 1234, 1272, 1499, 1551, 1642–1647, 1669, 1674, 1840, 1852, 2104, 2125, 2220, 2449, 2656, 2726, 2738, 2756, 2954–2956, 2959–2961, 2965, 2970, 2974, 3058–3060, 3089, 3143, 3162–3163, 3166–3170, 3176
   observability.py6588187%360, 362–364, 367–369, 374–375, 381–382, 388–389, 396, 398–400, 403–405, 410–411, 417–418, 424–425, 432, 470, 561, 703, 706, 714–715, 718–721, 723, 726–728, 731–732, 760, 762, 773–775, 777–779, 783, 791, 892, 894, 1043, 1045, 1049–1054, 1056, 1059–1063, 1065, 1174–1175, 1177, 1335, 1434, 1604, 1607, 1666, 1834, 1988, 1990
packages/core/agent_framework/azure
   _embedding_client.py210100% 
packages/core/agent_framework/openai
   _embedding_client.py55198%97
TOTAL21920277687% 

Python Unit Test Overview

Tests Skipped Failures Errors Time
4592 247 💤 0 ❌ 0 🔥 1m 15s ⏱️

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds embedding client implementations for Ollama, Bedrock, and Azure AI Inference as part of Phase 2 of the vector stores & embeddings port. It includes type variable renames, otel_provider_name passthrough additions to existing clients, pytest marker registration across packages, and lazy-loading namespace exports.

Changes:

  • Three new embedding clients: OllamaEmbeddingClient, BedrockEmbeddingClient, AzureAIInferenceEmbeddingClient with full test coverage
  • Type variable renames: EmbeddingCoTEmbeddingT, EmbeddingOptionsCoTEmbeddingOptionsT across core abstractions
  • Added otel_provider_name parameter to OpenAIEmbeddingClient and AzureOpenAIEmbeddingClient for telemetry customization
  • Registered integration pytest marker in all package pyproject.toml files for consistent test organization
  • Added azure-ai-inference>=1.0.0b9 dependency to azure-ai package

Reviewed changes

Copilot reviewed 40 out of 42 changed files in this pull request and generated no comments.

Show a summary per file
File Description
python/packages/ollama/agent_framework_ollama/_embedding_client.py New OllamaEmbeddingClient using Ollama's embed API with truncate/dimensions options
python/packages/bedrock/agent_framework_bedrock/_embedding_client.py New BedrockEmbeddingClient using Amazon Titan Embeddings via invoke_model API
python/packages/azure-ai/agent_framework_azure_ai/_embedding_client.py New AzureAIInferenceEmbeddingClient supporting both text and image embeddings with Content dispatching
python/packages/core/agent_framework/_clients.py Type variable renames EmbeddingCoT → EmbeddingT for consistency
python/packages/core/agent_framework/observability.py Type variable renames EmbeddingOptionsCoT → EmbeddingOptionsT
python/packages/core/agent_framework/openai/_embedding_client.py Added otel_provider_name parameter passthrough
python/packages/core/agent_framework/azure/_embedding_client.py Added otel_provider_name parameter passthrough
python/packages/*/pyproject.toml (multiple) Added integration pytest marker registration across 18 packages
python/packages/core/agent_framework/ollama/__init__.py Added lazy-loading exports for OllamaEmbeddingClient and related types
python/packages/core/agent_framework/amazon/__init__.py Added lazy-loading exports for BedrockEmbeddingClient and related types
python/samples/02-agents/embeddings/azure_ai_inference_embeddings.py Sample demonstrating image and mixed text+image embedding generation
python/uv.lock Dependency updates including azure-ai-inference and greenlet s390x removals

@eavanvalkenburg eavanvalkenburg force-pushed the feature/embedding-phase2 branch from 228b1ee to 03524a4 Compare February 24, 2026 13:02
eavanvalkenburg and others added 3 commits February 25, 2026 10:16
Add embedding client implementations to existing provider packages:

- OllamaEmbeddingClient: Text embeddings via Ollama's embed API
- BedrockEmbeddingClient: Text embeddings via Amazon Titan on Bedrock
- AzureAIInferenceEmbeddingClient: Text and image embeddings via Azure AI
  Inference, supporting Content | str input with separate model IDs for
  text (AZURE_AI_INFERENCE_EMBEDDING_MODEL_ID) and image
  (AZURE_AI_INFERENCE_IMAGE_EMBEDDING_MODEL_ID) endpoints

Additional changes:
- Rename EmbeddingCoT -> EmbeddingT, EmbeddingOptionsCoT -> EmbeddingOptionsT
- Add otel_provider_name passthrough to all embedding clients
- Register integration pytest marker in all packages
- Add lazy-loading namespace exports for Ollama and Bedrock embeddings
- Add image embedding sample using Cohere-embed-v3-english
- Add azure-ai-inference dependency to azure-ai package

Part of microsoft#1188

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Rename second 'vector' variable to 'img_vector' in image embedding loop
- Combine nested with statements in tests
- Remove unused result assignments in tests

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@eavanvalkenburg eavanvalkenburg force-pushed the feature/embedding-phase2 branch from 68c1dd6 to aa0dcbd Compare February 25, 2026 09:16
- Fix Azure AI embedding mypy issues by normalizing vectors to list[float],
  safely accumulating optional usage token fields, and filtering None entries
  before constructing GeneratedEmbeddings
- Avoid Bandit false positive by initializing usage details as an empty dict
- Update OpenAI embedding tests to assert canonical usage keys
  (input_token_count/total_token_count)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@eavanvalkenburg eavanvalkenburg added this pull request to the merge queue Feb 25, 2026
Merged via the queue into microsoft:main with commit 6138487 Feb 25, 2026
30 checks passed
westey-m added a commit that referenced this pull request Feb 26, 2026
* .NET: Add Microsoft Fabric sample #3674 (#4230)

Co-authored-by: Chris <66376200+crickman@users.noreply.github.com>

* Python: Phase 2: Embedding clients for Ollama, Bedrock, and Azure AI Inference (#4207)

* Phase 2: Embedding clients for Ollama, Bedrock, and Azure AI Inference

Add embedding client implementations to existing provider packages:

- OllamaEmbeddingClient: Text embeddings via Ollama's embed API
- BedrockEmbeddingClient: Text embeddings via Amazon Titan on Bedrock
- AzureAIInferenceEmbeddingClient: Text and image embeddings via Azure AI
  Inference, supporting Content | str input with separate model IDs for
  text (AZURE_AI_INFERENCE_EMBEDDING_MODEL_ID) and image
  (AZURE_AI_INFERENCE_IMAGE_EMBEDDING_MODEL_ID) endpoints

Additional changes:
- Rename EmbeddingCoT -> EmbeddingT, EmbeddingOptionsCoT -> EmbeddingOptionsT
- Add otel_provider_name passthrough to all embedding clients
- Register integration pytest marker in all packages
- Add lazy-loading namespace exports for Ollama and Bedrock embeddings
- Add image embedding sample using Cohere-embed-v3-english
- Add azure-ai-inference dependency to azure-ai package

Part of #1188

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix mypy duplicate name and ruff lint issues

- Rename second 'vector' variable to 'img_vector' in image embedding loop
- Combine nested with statements in tests
- Remove unused result assignments in tests

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* updates from feedback

* Fix CI failures in embedding usage handling

- Fix Azure AI embedding mypy issues by normalizing vectors to list[float],
  safely accumulating optional usage token fields, and filtering None entries
  before constructing GeneratedEmbeddings
- Avoid Bandit false positive by initializing usage details as an empty dict
- Update OpenAI embedding tests to assert canonical usage keys
  (input_token_count/total_token_count)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* [Purview] Mark responses as responses and fix epoch bug for python long overflow (#4225)

* .NET: Support InvokeMcpTool for declarative workflows (#4204)

* Initial implementation of InvokeMcpTool in declarative workflow

* Cleaned up sample implementation

* Updated sample comments.

* Added missing executor routing attribute

* Fix PR comments.

* Updated based on PR comments.

* Updated based on PR comments.

* Removed unnecessary using statement.

* Update Python package versions to rc2 (#4258)

- Bump core and azure-ai to 1.0.0rc2
- Bump preview packages to 1.0.0b260225
- Update dependencies to >=1.0.0rc2
- Add CHANGELOG entries for changes since rc1
- Update uv.lock

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* .NET: Fixing issue where OpenTelemetry span is never exported in .NET in-process workflow execution (#4196)

* 1. Add reproduction test for issue #4155: workflow.run Activity never stopped in streaming OffThread path

The WorkflowRunActivity_IsStopped_Streaming_OffThread test demonstrates that
the workflow.run OpenTelemetry Activity created in StreamingRunEventStream.RunLoopAsync
is started but never stopped when using the OffThread/Default streaming execution.
The background run loop keeps running after event consumption completes, so the
using Activity? declaration never disposes until explicit StopAsync() is called.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

2. Fix workflow.run Activity never stopped in streaming OffThread execution (#4155)

The workflow.run OpenTelemetry Activity in StreamingRunEventStream.RunLoopAsync
was scoped to the method lifetime via 'using'. Since the run loop only exits on
cancellation, the Activity was never stopped/exported until explicit disposal.

Fix: Remove 'using' and explicitly dispose the Activity when the workflow reaches
Idle status (all supersteps complete). A safety-net disposal in the finally block
handles cancellation and error paths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add root-level workflow.session activity spanning run loop lifetime\n\nImplements two-level telemetry hierarchy per PR feedback from lokitoth:\n- workflow.session: spans the entire run loop / stream lifetime\n- workflow_invoke: per input-to-halt cycle, nested within the session\n\nThis ensures the session activity stays open across multiple turns,\nwhile individual run activities are created and disposed per cycle.\n\nAlso fixes linkedSource CancellationTokenSource disposal leak in\nStreamingRunEventStream (added using declaration)."

* Address Copilot review: fix Activity/CTS disposal, rename activity, add error tag\n\n1. LockstepRunEventStream: Remove 'using' from Activity in async iterator\n   and manually dispose in finally block (fixes #4155 pattern). Also dispose\n   linkedSource CTS in finally to prevent leak.\n2. Tags.cs: Add ErrorMessage (\"error.message\") tag for runtime errors,\n   distinct from BuildErrorMessage (\"build.error.message\").\n3. ActivityNames: Rename WorkflowRun from \"workflow_invoke\" to \"workflow.run\"\n   for cross-language consistency.\n4. WorkflowTelemetryContext: Fix XML doc to say \"outer/parent span\" instead\n   of \"root-level span\".\n5. ObservabilityTests: Assert WorkflowSession absence when DisableWorkflowRun\n   is true.\n6. WorkflowRunActivityStopTests: Fix streaming test race by disposing\n   StreamingRun before asserting activities are stopped.\n7. StreamingRunEventStream/LockstepRunEventStream: Use Tags.ErrorMessage\n   instead of Tags.BuildErrorMessage for runtime error events."

* Review fixes: revert workflow_invoke rename, use 'using' for linkedSource, move SessionStarted earlier\n\n- Revert ActivityNames.WorkflowRun back to \"workflow_invoke\" (OTEL semantic convention contract)\n- Use 'using' declaration for linkedSource CTS in LockstepRunEventStream (no timing sensitivity)\n- Move SessionStarted event before WaitForInputAsync in StreamingRunEventStream to match Lockstep behavior"

* Improve naming and comments in WorkflowRunActivityStopTests"

* Prevent session Activity.Current leak in lockstep mode, add nesting test

Save and restore Activity.Current in LockstepRunEventStream.Start() so the
session activity doesn't leak into caller code via AsyncLocal. Re-establish
Activity.Current = sessionActivity before creating the run activity in
TakeEventStreamAsync to preserve parent-child nesting.

Add test verifying app activities after RunAsync are not parented under the
session, and that the workflow_invoke activity nests under the session."

* Fix stale XML doc: WorkflowRun -> WorkflowInvoke in ObservabilityTests

---------

Co-authored-by: alliscode <bentho@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python / .NET Samples - Restructure and Improve Samples (Feature Branc… (#4092)

* Python: .NET Samples - Restructure and Improve Samples (Feature Branch) (#4091)

* Moved by agent (#4094)

* Fix readme links

* .NET Samples - Create `04-hosting` learning path step (#4098)

* Agent move

* Agent reorderd

* Remove A2A section from README 

Removed A2A section from the Getting Started README.

* Agent fixed links

* Fix broken sample links in durable-agents README (#4101)

* Initial plan

* Fix broken internal links in documentation

Co-authored-by: crickman <66376200+crickman@users.noreply.github.com>

* Revert template link changes; keep only durable-agents README fix

Co-authored-by: crickman <66376200+crickman@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: crickman <66376200+crickman@users.noreply.github.com>

* .NET Samples - Create `03-workflows` learning path step (#4102)

* Fix solution project path

* Python: Fix broken markdown links to repo resources (outside /docs) (#4105)

* Initial plan

* Fix broken markdown links to repo resources

Co-authored-by: crickman <66376200+crickman@users.noreply.github.com>

* Update README to rename .NET Workflows Samples section

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: crickman <66376200+crickman@users.noreply.github.com>

* .NET Samples - Create `02-agents` learning path step (#4107)

* .NET: Fix broken relative link in GroupChatToolApproval README (#4108)

* Initial plan

* Fix broken link in GroupChatToolApproval README

Co-authored-by: crickman <66376200+crickman@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: crickman <66376200+crickman@users.noreply.github.com>

* Update labeler configuration for workflow samples

* .NET - Reorder Agents samples to start from Step01 instead of Step04 (#4110)

* Fix solution

* Resolve new sample paths

* Move new AgentSkills and AgentWithMemory_Step04 samples

* Fix link

* Fix readme path

* fix: update stale dotnet/samples/Durable path reference in AGENTS.md

Co-authored-by: crickman <66376200+crickman@users.noreply.github.com>

* Moved new sample

* Update solution

* Resolve merge (new sample)

* Sync to new sample - FoundryAgents_Step21_BingCustomSearch

* Updated README

* .NET Samples - Configuration Naming Update (#4149)

* .NET: Restore AzureFunctions index parity with ConsoleApps under DurableAgents samples (#4221)

* Clean-up `05_host_your_agent`

* Config setting consistency

* Refine samples

* AGENTS.md

* Move new samples

* Re-order samples

* Move new project and fixup solution

* Fixup model config

* Fix up new UT project

---------

Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>

* Python: Fix Bedrock embedding test stub missing meta attribute (#4287)

* Fix Bedrock embedding test stub missing meta attribute

* Increase test coverage so gate passes

* Python: (ag-ui): fix approval payloads being re-processed on subsequent conversation turns (#4232)

* Fix ag-ui tool call issue

* Safe json fix

* Python: Update workflow orchestration samples to use AzureOpenAIResponsesClient (#4285)

* Update workflow orchestration samples to use AzureOpenAIResponsesClient

* Fix broken link

* Move scripts to scripts folder

---------

Co-authored-by: Roger Barreto <19890735+rogerbarreto@users.noreply.github.com>
Co-authored-by: Chris <66376200+crickman@users.noreply.github.com>
Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Rishabh Chawla <rishabhchawla1995@gmail.com>
Co-authored-by: Peter Ibekwe <109177538+peibekwe@users.noreply.github.com>
Co-authored-by: Dmytro Struk <13853051+dmytrostruk@users.noreply.github.com>
Co-authored-by: Ben Thomas <ben.thomas@microsoft.com>
Co-authored-by: alliscode <bentho@microsoft.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Evan Mattson <35585003+moonbox3@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: Phase 2: Embedding Generators for Existing Providers

6 participants