.NET: Python: Native support for Bedrock-hosted models (Anthropic, Cohere, etc.) in Python SDK by duttastunil · Pull Request #2560 · microsoft/agent-framework

duttastunil · 2025-12-01T20:52:47Z

Motivation and Context

The current Python SDK for the Microsoft Agent Framework requires users to route requests to Bedrock models via the generic OpenAI-compatible API interface. This approach has several drawbacks:

It does not natively recognize specific model IDs (e.g., anthropic.claude-3-5-sonnet-20240620-v1:0).
It may lack native support for provider-specific parameters or functionality (like Anthropic's specific messaging API structure, context windows, or token cost tracking).
It creates a less intuitive configuration experience compared to native integrations for Azure OpenAI or generic OpenAI services.
I would like to request dedicated, native client classes and configuration support within the Python SDK to directly integrate models hosted on Amazon Bedrock.

Description

Below are the details of the BedrockChatClient:

Async implementation of the Agent Framework BaseChatClient that targets Amazon Bedrock’s Converse API.
Handles provider setup: reads Bedrock credentials/region from env (via BedrockSettings), builds a boto3 bedrock-runtime client with the AF user-agent, and exposes service_url.
Converts framework messages and tools into Bedrock-compatible payloads (system prompts, content blocks, tool specs/choices) while enforcing Bedrock request rules (e.g., aligning tool results with pending tool calls).
Issues converse calls and adapts results back into Agent Framework structures: ChatResponse, FunctionCallContent, FunctionResultContent, usage metrics, and finish reasons.
Supports streaming via _inner_get_streaming_response, automatically adds tool usage telemetry (use_function_invocation), middleware hooks, and OpenTelemetry metadata (use_observability).

Contribution Checklist

The code builds clean without any errors or warnings
The PR follows the Contribution Guidelines
All unit tests pass, and I have added new tests where possible
This is not a breaking change

eavanvalkenburg

some comments on the code.
The folder itself should be python/packages/bedrock with agent_framework_bedrock inside, next to the pyproject and other files.
And we are missing creating a folder inside the core package called amazon that lazy loads from agent_framework_bedrock (look at the Anthropic folder for how)

eavanvalkenburg · 2025-12-02T08:43:20Z

python/packages/agent_framework.amazon/pyproject.toml

+
+[project.optional-dependencies]
+dev = [
+    "pytest>=8.3.0",


we already install pytest as a dev dependency

eavanvalkenburg · 2025-12-02T08:44:12Z

python/samples/amazon/bedrock_sample.py

this should just be samples/amazon/<sample name>.py

eavanvalkenburg · 2025-12-02T08:44:19Z

python/samples/agent_framework.amazon/agent_framework_bedrock/main.py

@@ -0,0 +1,63 @@
+from __future__ import annotations


is this necessary?

eavanvalkenburg · 2025-12-02T08:44:31Z

python/samples/agent_framework.amazon/agent_framework_bedrock/main.py

+    ToolMode,
+    ai_function,
+)
+from agent_framework_bedrock import BedrockChatClient


Suggested change

from agent_framework_bedrock import BedrockChatClient

from agent_framework.amazon import BedrockChatClient

eavanvalkenburg · 2025-12-02T08:44:58Z

python/packages/agent_framework.amazon/agent_framework_bedrock/_chat_client.py

@@ -0,0 +1,504 @@
+# Copyright (c) Microsoft. All rights reserved.
+
+from __future__ import annotations


we shouldn't need this.

eavanvalkenburg · 2025-12-02T08:45:42Z

python/packages/agent_framework.amazon/agent_framework_bedrock/_chat_client.py

+        access_key: str | None = None,
+        secret_key: str | None = None,
+        session_token: str | None = None,
+        bedrock_runtime_client: BaseClient | None = None,


for consistency use:

Suggested change

bedrock_runtime_client: BaseClient | None = None,

client: BaseClient | None = None,

eavanvalkenburg · 2025-12-02T08:47:12Z

python/packages/bedrock/agent_framework_bedrock/_chat_client.py

+        **kwargs: Any,
+    ) -> ChatResponse:
+        request = self._build_converse_request(messages, chat_options, **kwargs)
+        raw_response = await asyncio.to_thread(self._bedrock_client.converse, **request)


is there no native async method?

AWS’s Bedrock runtime SDK is exposed only through boto3/botocore, which are synchronous clients. There isn’t an official async interface (no converse_async or async-aware session) today.

eavanvalkenburg · 2025-12-02T08:47:49Z

python/packages/bedrock/agent_framework_bedrock/_chat_client.py

+        chat_options: ChatOptions,
+        **kwargs: Any,
+    ) -> AsyncIterable[ChatResponseUpdate]:
+        response = await self._inner_get_response(messages=messages, chat_options=chat_options, **kwargs)


and no streaming method?

I think AWS has a ConverseStream API in preview for select providers, but it isn’t in the generally available boto3 client yet, and it doesn’t cover every model. Until the official SDK exposes a stable streaming method, we can’t add a native streaming implementation. The current shim keeps our interface consistent and can be upgraded once AWS ships wider support.

eavanvalkenburg · 2025-12-02T08:48:57Z

python/packages/agent_framework.amazon/agent_framework_bedrock/_chat_client.py

+            payload["system"] = system_prompts
+
+        inference_config: dict[str, Any] = {}
+        if chat_options.max_tokens is not None:


Suggested change

if chat_options.max_tokens is not None:

inference_config["maxTokens"] = chat_options.max_tokens if chat_options.max_tokens is not None else DEFAULT_MAX_TOKENS

eavanvalkenburg · 2025-12-02T08:50:43Z

python/packages/agent_framework.amazon/agent_framework_bedrock/_chat_client.py

+                "Bedrock model_id is required. Set via chat options or BEDROCK_CHAT_MODEL_ID environment variable."
+            )
+
+        system_prompts = self._extract_system_prompts(messages)


in these two functions, we are looping through the messages twice, maybe combine and return the first part of the payload

duttastunil · 2025-12-03T14:27:15Z

@eavanvalkenburg - I addressed all the comments on this PR, Please have a look on the latest code/PR. Thanks!!

duttastunil · 2025-12-03T14:31:27Z

Updated python/packages/core/agent_framework/_workflows/_validation.py due to the below issue.

Found 1 error.
No fixes available (1 hidden fix can be enabled with the --unsafe-fixes option).
Error: Sequence aborted after failed subtask 'pre-commit-check[1]'

duttastunil · 2025-12-03T17:38:12Z

@eavanvalkenburg - Since I ran precommit check command and other commands on my local, it seems multiple file changed,
I created a new PR just to keep the PR clean for review. I addressed all the review comments in this PR, Please have a look.

https://github.com/microsoft/agent-framework/pull/2610/files

2524 - added the bedrock hosted models integration changes

a54dee1

markwallace-microsoft added documentation Improvements or additions to documentation python labels Dec 1, 2025

eavanvalkenburg reviewed Dec 2, 2025

View reviewed changes

2524 Addressed the review comments on PR

6472dc0

Addressed the review comments and resolved the branch conflict

47c31c0

duttastunil requested a review from a team as a code owner December 3, 2025 17:31

markwallace-microsoft added the .NET label Dec 3, 2025

github-actions bot changed the title ~~Python: Native support for Bedrock-hosted models (Anthropic, Cohere, etc.) in Python SDK~~ .NET: Python: Native support for Bedrock-hosted models (Anthropic, Cohere, etc.) in Python SDK Dec 3, 2025

eavanvalkenburg closed this Dec 9, 2025

	from agent_framework_bedrock import BedrockChatClient
	from agent_framework.amazon import BedrockChatClient

		@@ -0,0 +1,504 @@
		# Copyright (c) Microsoft. All rights reserved.

		from __future__ import annotations

	bedrock_runtime_client: BaseClient \| None = None,
	client: BaseClient \| None = None,

	if chat_options.max_tokens is not None:
	inference_config["maxTokens"] = chat_options.max_tokens if chat_options.max_tokens is not None else DEFAULT_MAX_TOKENS

Conversation

duttastunil commented Dec 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation and Context

Description

Contribution Checklist

Uh oh!

eavanvalkenburg left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

duttastunil commented Dec 3, 2025

Uh oh!

duttastunil commented Dec 3, 2025

Uh oh!

duttastunil commented Dec 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

duttastunil commented Dec 1, 2025 •

edited

Loading

duttastunil commented Dec 3, 2025 •

edited

Loading