Python: [BREAKING] Moved to a single get_response and run API by eavanvalkenburg · Pull Request #3379 · microsoft/agent-framework

eavanvalkenburg · 2026-01-22T17:34:18Z

Motivation and Context

Summary

Migrate chat/agent telemetry to mixin-based usage and remove legacy decorators, with streaming telemetry now using finalizers/teardown hooks instead of consuming streams.
- This makes understanding the code a lot simpler, because we can set attributes on the chat client in the init of those mixin (making them technically not a mixin)
- Added those parameters to the constructors, making it easier to configure things like function calling
Replace function invocation decorators with FunctionInvokingChatClient/FunctionInvokingMixin across clients, tests, and samples; update docs/comments accordingly.
Introducing a ResponseStream object that can is created to unify the API's
- It is generic over TUpdate and TFinal, which in our case is usually ChatResponseUpdate and ChatReponse or the agent equivalent.
- It features a update_hook mechanism, to allow you to run code while the internal stream is being unpacked, this can mostly be leveraged by middleware
- It features a teardown hook mechanism, this get's run when the stream is exhausted, it's used now by the telemtry to record the duration
- It features a finalizer (one or more) mechanism, that runs after the end of the stream, which is used to turn the updates list into a final object, this can be used by middleware and is also used in function calling and telemetry
- In principle the ResponseStream is created by the most lowlevel object, the actual chat client implementations, and ideally all the layers in between should only use the hooks to do something, FunctionCalling does not work that way, because there are multiple calls to the underlying chat client that all then have to be combined into a single stream at runtime. Agent also creates a new stream, because it goes from ResponseStream[ChatResponseUpdate, ChatResponse] to ResponseStream[AgentResponseUpdate, AgentResponse], but the object has a classmethod called wrap that is used to wrap the ResponseStream from the chat client into the new ResponseStream in the Agent.
Overall this change reduces the number of times we iterate the stream and return a new AsyncGenerator and the new hooks actually make it simpler to create middleware that alters the stream (as the sample shows), it should therefore also improve performance a bit.
Removed use_instrumentation/use_agent_instrumentation and use_function_invocation decorators; mixins are now the supported path.

Description

Contribution Checklist

The code builds clean without any errors or warnings
The PR follows the Contribution Guidelines
All unit tests pass, and I have added new tests where possible
Is this a breaking change? If yes, add [BREAKING] prefix to the title of the PR.

Fixes #3585
Fixes #3607
Fixes #3617
Fixes #3581
Fixes #3610

Copilot

Pull request overview

This PR consolidates the Python Agent Framework's streaming and non-streaming APIs into a unified interface. The primary changes include:

Changes:

Unified run() and get_response() methods with stream parameter replacing separate run_stream() and get_streaming_response() methods
Migration from decorator-based (@use_instrumentation, @use_function_invocation) to mixin-based architecture for telemetry and function invocation
Introduction of ResponseStream class for unified stream handling with hooks, finalizers, and teardown support
Renamed AgentExecutionException to AgentRunException

Reviewed changes

Copilot reviewed 84 out of 85 changed files in this pull request and generated 28 comments.

Show a summary per file

File	Description
`_types.py`	Added `ResponseStream` class for unified streaming, updated `prepare_messages` to handle None
`_clients.py`	Refactored `BaseChatClient` with unified `get_response()` method, introduced `FunctionInvokingChatClient` mixin
`openai/_responses_client.py`	Consolidated streaming/non-streaming into single `_inner_get_response()` method
`openai/_chat_client.py`	Similar consolidation for chat completions API
`openai/_assistants_client.py`	Unified assistants API with stream parameter
`_workflows/_workflow.py`	Consolidated `run()` and `run_stream()` into single `run(stream=bool)` method
`_workflows/_agent.py`	Updated `WorkflowAgent.run()` to use stream parameter
Test files (multiple)	Updated all tests to use `run(stream=True)` and `get_response(stream=True)`
Sample files (multiple)	Updated samples to demonstrate new unified API
Provider clients	Updated all provider implementations (Azure, Anthropic, Bedrock, Ollama, etc.) to use mixins

python/packages/core/agent_framework/openai/_responses_client.py

python/packages/core/agent_framework/openai/_assistants_client.py

python/packages/core/agent_framework/_clients.py

python/packages/core/agent_framework/exceptions.py

python/packages/core/agent_framework/observability.py

python/packages/core/tests/workflow/test_agent_executor_tool_calls.py

python/packages/core/tests/workflow/test_handoff.py

python/packages/core/agent_framework/_tools.py

markwallace-microsoft · 2026-01-23T11:16:58Z

Python Test Coverage Report •

File	Stmts	Miss	Cover	Missing
packages/a2a/agent_framework_a2a
_agent.py	148	8	94%	261, 399–400, 437–438, 467–469
packages/ag-ui/agent_framework_ag_ui
_client.py	150	17	88%	83–84, 88–92, 96–100, 263, 295, 464–466
_message_adapters.py	454	95	79%	89, 99–100, 109–112, 115–119, 121–126, 129, 138–144, 147, 151–153, 162–164, 184, 190–192, 222, 235–236, 246–247, 284, 287, 289, 292, 295, 311, 328, 350, 381, 386, 397–398, 449, 465–466, 532–535, 537, 543, 551–552, 554, 558–561, 574, 663–666, 668, 734, 769–771, 773–776, 779–780, 782, 788, 791, 793, 796, 798, 804–805, 807
_run.py	481	121	74%	155–162, 305, 324–325, 340–341, 356, 384–386, 411, 414–417, 419–420, 423–429, 432–434, 437, 453–455, 462, 468–470, 474, 479–481, 483–484, 500–504, 515, 528, 530–531, 547, 568–569, 626–628, 640–642, 840, 851–852, 859, 877–879, 913–915, 932, 938, 946, 948, 984–990, 993–996, 998–1007, 1010, 1018–1021, 1028, 1031–1032, 1037, 1043–1045, 1049, 1054–1057, 1071–1073
_types.py	36	0	100%
_utils.py	101	3	97%	72, 257, 262
packages/ag-ui/agent_framework_ag_ui/_orchestration
_tooling.py	57	0	100%
packages/anthropic/agent_framework_anthropic
_chat_client.py	362	150	58%	371, 403, 405, 420, 442–445, 454, 456, 487–491, 493, 495–496, 498, 503–504, 506, 539–540, 549, 551–552, 557, 574–575, 617, 632, 636–637, 653, 662, 664, 668–669, 712–714, 716, 729–730, 737–739, 743–745, 749–752, 763, 765, 787, 797, 819–825, 832–833, 841–842, 850–853, 860–861, 867–868, 874–875, 881, 889–891, 895, 902–903, 909–910, 916–917, 923, 931–934, 941–942, 961, 968–969, 988, 1010, 1012, 1021–1022, 1028, 1050–1051, 1057–1058, 1067–1077, 1084–1090, 1097–1103, 1110–1119, 1126–1129
packages/azure-ai/agent_framework_azure_ai
_agent_provider.py	115	3	97%	122–123, 251
_chat_client.py	484	75	84%	382, 387–388, 390–391, 394, 397, 399, 404, 665–666, 668, 671, 674, 677–682, 685, 687, 695, 707–709, 713, 716–717, 725–728, 738, 746–749, 751–752, 754–755, 762, 770–771, 779–780, 785–786, 790–797, 802, 805, 813, 819, 827–829, 832, 854–855, 988, 1016, 1031, 1152, 1178, 1187, 1196, 1329
_client.py	195	13	93%	360, 362, 411, 440–445, 488, 523, 525, 601
_project_provider.py	115	6	94%	132–133, 211, 309, 353, 386
packages/chatkit/agent_framework_chatkit
_converter.py	133	46	65%	115, 120, 168, 170, 340, 393, 395, 414–416, 418, 436, 438, 440, 443, 455, 465, 483, 503–527, 529–531
packages/copilotstudio/agent_framework_copilotstudio
_agent.py	83	5	93%	154–155, 190, 198, 315
packages/core/agent_framework
_agents.py	320	35	89%	473, 885, 921, 1020–1022, 1135, 1176, 1178, 1187–1192, 1198, 1200, 1210–1211, 1218, 1220–1221, 1229–1233, 1241–1242, 1244, 1249, 1251, 1285, 1325, 1345
_clients.py	52	3	94%	294, 495, 497
_middleware.py	329	16	95%	80, 83, 88, 797, 799, 801, 922, 949, 951, 976, 1057, 1061, 1185, 1189, 1250, 1324
_serialization.py	105	4	96%	516, 532, 542, 610
_threads.py	136	3	97%	343, 474–475
_tools.py	788	85	89%	232, 278, 329, 331, 359, 529, 564–565, 667, 669, 689, 707, 721, 733, 738, 740, 747, 780, 851–853, 894, 916–944, 979, 987, 1228, 1433, 1490, 1494, 1573–1577, 1595, 1597–1598, 1703, 1707, 1757, 1759, 1775, 1777, 1841, 1868, 1921, 1989, 2165–2166, 2193, 2201, 2214, 2224–2225, 2260, 2316, 2348
_types.py	1038	96	90%	81, 90–91, 145, 150, 169, 171, 175, 179, 181, 183, 185, 203, 207, 233, 255, 260, 265, 269, 295, 299, 645–646, 1017, 1079, 1096, 1114, 1119, 1137, 1147, 1164–1165, 1167, 1185–1186, 1188, 1195–1196, 1198, 1233, 1244–1245, 1247, 1285, 1542, 1595, 1602, 1624, 1630, 1678, 1721–1726, 1748, 1753, 1882, 1894, 2133, 2142, 2162, 2254, 2475, 2743, 2747, 2759, 2766, 2777, 2981–2983, 2986–2988, 2992, 2997, 3001, 3113–3115, 3143, 3197, 3201–3203, 3205, 3216–3217, 3220–3224, 3230
observability.py	607	84	86%	334, 336–338, 341–343, 348–349, 355–356, 362–363, 370, 372–374, 377–379, 384–385, 391–392, 398–399, 406, 662, 665, 673–674, 677–680, 682, 685–687, 690–691, 719, 721, 732–734, 736–739, 743, 751, 852, 854, 1003, 1005, 1009–1014, 1016, 1019–1023, 1025, 1137–1138, 1140, 1191–1192, 1327, 1375–1376, 1492–1494, 1553, 1723, 1877, 1879
packages/core/agent_framework/_workflows
_agent.py	332	54	83%	59, 67–73, 107–108, 340–341, 350–351, 358, 360, 366–367, 453–454, 463, 470, 496, 547, 576, 635–638, 644, 650, 654–655, 658–664, 668–669, 675, 749–750, 761, 800, 821, 830, 834, 836–838, 845
_agent_executor.py	160	22	86%	98, 146, 164–165, 216–217, 219–220, 252–254, 262–264, 274–276, 278, 282, 286, 290–291
_base_group_chat_orchestrator.py	170	12	92%	135, 301, 316, 350–352, 356, 375, 436, 480–482
_const.py	6	0	100%
_conversation_state.py	36	5	86%	40, 45, 47, 49, 64
_message_utils.py	18	3	83%	22, 33, 37
_orchestration_request_info.py	54	0	100%
_orchestrator_helpers.py	21	2	90%	90–91
_runner_context.py	168	6	96%	84, 87, 383, 403, 491, 495
_workflow.py	272	19	93%	89, 267–269, 271–272, 290, 294, 322, 424, 642, 684, 689, 692, 711–713, 726, 794
_workflow_context.py	157	9	94%	62–63, 71, 75, 89, 165, 190, 308, 427
packages/core/agent_framework/azure
_chat_client.py	79	4	94%	301, 303, 316–317
_responses_client.py	37	6	83%	146, 169, 198–201
packages/core/agent_framework/openai
_assistant_provider.py	110	11	90%	156–157, 169, 294, 360, 475–480
_assistants_client.py	275	35	87%	358, 360, 362, 365, 369–370, 373, 376, 381–382, 384, 387–389, 394, 405, 430, 432, 434, 436, 438, 443, 446, 449, 453, 464, 549, 634, 671, 708–711, 763, 780
_chat_client.py	267	21	92%	179–180, 184, 298, 305, 386–393, 395–398, 408, 493, 530, 546
_responses_client.py	562	62	88%	277–278, 283, 314, 322, 345, 407, 439, 464, 470, 488–489, 511, 516, 572, 586, 603, 616, 671, 750, 755, 759–761, 765–766, 789, 858, 880–881, 896–897, 915–916, 1047–1048, 1064, 1066, 1141–1149, 1197, 1252, 1267, 1303–1304, 1306–1308, 1322–1324, 1334–1335, 1341, 1356
_shared.py	135	16	88%	63, 69–72, 151, 153, 155, 162, 164, 177, 253, 277, 341–342, 344
packages/mem0/agent_framework_mem0
_provider.py	85	6	92%	67–68, 90, 174–175, 178
packages/orchestrations/agent_framework_orchestrations
_group_chat.py	291	39	86%	171, 332, 339, 366, 377–378, 384, 389, 405, 432–437, 439, 472–475, 477, 482–486, 650, 655, 669, 750, 756, 802, 822, 897–898, 932, 951, 970, 980
_handoff.py	383	57	85%	109–110, 112, 141–142, 164–174, 176, 178, 180, 185, 285, 339, 364, 392, 400–401, 415, 464–465, 497, 544–546, 733, 740, 745, 832, 835, 844–847, 857, 862, 869, 875–878, 913, 918, 1115, 1118, 1126, 1144, 1151, 1226
_magentic.py	620	91	85%	66–75, 80, 84–95, 260, 271, 275, 295, 356, 365, 367, 409, 426, 435–436, 438–440, 442, 453, 595, 597, 637, 685, 721–723, 725, 733–736, 740–743, 786, 813–816, 907, 913, 919, 958, 996, 1025, 1042, 1053, 1107–1108, 1112–1114, 1138, 1159–1160, 1173, 1189, 1211, 1259–1260, 1298–1299, 1458, 1467, 1470, 1475, 1871, 1926, 1941, 1970
packages/purview/agent_framework_purview
_middleware.py	95	0	100%
packages/redis/agent_framework_redis
_chat_message_store.py	149	14	90%	199, 232, 322–323, 326, 329, 485, 575–579, 588, 592
_provider.py	188	9	95%	255, 257, 265, 270–271, 274, 329, 386, 398
TOTAL	16514	2008	87%

Python Unit Test Overview

Tests	Skipped	Failures	Errors	Time
3910	225 💤	0 ❌	0 🔥	1m 5s ⏱️

python/packages/a2a/agent_framework_a2a/_agent.py

python/packages/anthropic/agent_framework_anthropic/_chat_client.py

python/packages/azure-ai/agent_framework_azure_ai/_client.py

python/packages/core/agent_framework/_workflows/_agent_executor.py

python/packages/core/agent_framework/_agents.py

python/packages/core/agent_framework/observability.py

eavanvalkenburg · 2026-02-02T10:26:04Z

Fixes lingering CI failures: import missing response types in streaming telemetry finalizers, move AG-UI tests to ag_ui_tests with config updates, and track service thread IDs in AG-UI test client.\n\nChecks: uv run poe fmt/lint/pyright/mypy; uv run poe all-tests.

python/packages/core/agent_framework/observability.py

python/packages/core/agent_framework/_tools.py

…types

- Refactor streaming function invocation to use get_final_response() on inner streams - Fix MiddlewareTermination to accept result parameter for passing results - Fix _AutoHandoffMiddleware to use MiddlewareTermination instead of context.terminate - Fix AgentMiddlewareLayer.run() to properly forward function/chat middleware - Remove duplicate middleware registration in AgentMiddlewareLayer.__init__ - Fix exception handling in _auto_invoke_function to properly capture termination - Fix mypy errors in core package - Update tests to use stream=True parameter for unified run API

- Merge testutils.py into conftest.py for azurefunctions integration tests - Merge dt_testutils.py into conftest.py for durabletask integration tests - Convert all integration tests to use fixtures instead of direct imports (fixes ModuleNotFoundError with --import-mode=importlib) - Add sample_helper fixture for azurefunctions tests - Add agent_client_factory and orchestration_helper fixtures for durabletask - Integration tests now skip with descriptive messages when services unavailable - Restructure devui tests into tests/devui/ with proper conftest.py - Add test organization guidelines to CODING_STANDARD.md - Remove __init__.py from test directories per pytest best practices

The hook was skipping all tests in the test session, not just integration tests. Now it only skips items in the integration_tests directory.

@patch

Use patch.object on the imported module instead of @patch with string path to ensure the mock takes effect regardless of import timing.

Increase from 2 to 8 seconds to allow time for: - Python startup and module imports - Azure OpenAI client creation - Agent registration with DTS worker - Worker connection to DTS This helps prevent test failures in CI where the first tests may run before the worker is fully ready to process requests.

The _consume_stream method now expects a ResponseStream that can provide a final AgentResponse via get_final_response(). Update the test to use ResponseStream with AgentResponse.from_updates as the finalizer.

…ples

Copilot AI review requested due to automatic review settings January 22, 2026 17:34

markwallace-microsoft added documentation Improvements or additions to documentation python labels Jan 22, 2026

Copilot started reviewing on behalf of eavanvalkenburg January 22, 2026 17:35 View session

Copilot AI reviewed Jan 22, 2026

View reviewed changes

eavanvalkenburg force-pushed the python_single_response branch 3 times, most recently from 07afd46 to dd65afa Compare January 23, 2026 10:46

eavanvalkenburg changed the title ~~Python: [BREAKING} Python single response~~ Python: [BREAKING] Moved to a single get_response and run API Jan 23, 2026

eavanvalkenburg force-pushed the python_single_response branch 4 times, most recently from 32f0473 to 5c78d91 Compare January 30, 2026 05:03

markwallace-microsoft mentioned this pull request Jan 30, 2026

Python: NET: [Feature] Simplify AIAgent run methods #3515

Open

eavanvalkenburg requested a review from a team as a code owner January 30, 2026 16:25

TaoChenOSU reviewed Jan 30, 2026

View reviewed changes

markwallace-microsoft added .NET workflows Related to Workflows in agent-framework lab Agent Framework Lab labels Feb 1, 2026

github-actions bot changed the title ~~Python: [BREAKING] Moved to a single get_response and run API~~ .NET: Python: [BREAKING] Moved to a single get_response and run API Feb 1, 2026

eavanvalkenburg force-pushed the python_single_response branch from ebfc3b0 to 92995e6 Compare February 1, 2026 14:58

eavanvalkenburg removed the .NET label Feb 1, 2026

eavanvalkenburg changed the title ~~.NET: Python: [BREAKING] Moved to a single get_response and run API~~ Python: [BREAKING] Moved to a single get_response and run API Feb 1, 2026

moonbox3 reviewed Feb 3, 2026

View reviewed changes

python/packages/core/agent_framework/observability.py Outdated Show resolved Hide resolved

python/packages/core/agent_framework/observability.py Outdated Show resolved Hide resolved

python/packages/core/agent_framework/observability.py Outdated Show resolved Hide resolved

eavanvalkenburg force-pushed the python_single_response branch from a8f7c92 to a5dadf8 Compare February 3, 2026 15:36

dmytrostruk approved these changes Feb 4, 2026

View reviewed changes

python/packages/core/agent_framework/_tools.py Outdated Show resolved Hide resolved

eavanvalkenburg force-pushed the python_single_response branch from 92df8e3 to a99fdba Compare February 4, 2026 08:08

eavanvalkenburg enabled auto-merge February 4, 2026 08:23

eavanvalkenburg added 16 commits February 5, 2026 20:18

fixed issue in tests

0aeb6c3

fix: resolve merge conflict markers in ag-ui tests

278c7a1

fix: restructure ag-ui tests and fix Role/FinishReason to use string …

791d208

…types

fix all tests command

07b699a

Fix pytest_collection_modifyitems to only skip integration tests

5a83cfb

The hook was skipping all tests in the test session, not just integration tests. Now it only skips items in the integration_tests directory.

Fix mem0 tests failing on Python 3.13

67d9ca5

Use patch.object on the imported module instead of @patch with string path to ensure the mock takes effect regardless of import timing.

fix mem0

49fe1e5

another attempt for mem0

d9552e0

fix for mem0

e82a7ab

fix mem0

67ec9f5

Fix streaming test to use ResponseStream with finalizer

5f4d3cf

The _consume_stream method now expects a ResponseStream that can provide a final AgentResponse via get_final_response(). Update the test to use ResponseStream with AgentResponse.from_updates as the finalizer.

Fix MockToolCallingAgent to use new ResponseStream API and update sam…

2c3545a

…ples

small updates to run_stream to run

7990238

eavanvalkenburg force-pushed the python_single_response branch from 0da7fb1 to 7990238 Compare February 5, 2026 19:19

fix sub workflow

e370e22

eavanvalkenburg enabled auto-merge February 5, 2026 19:32

dmytrostruk approved these changes Feb 5, 2026

View reviewed changes

eavanvalkenburg added this pull request to the merge queue Feb 5, 2026

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Feb 5, 2026

temp fix for az func test

6107218

eavanvalkenburg enabled auto-merge February 5, 2026 19:49

dmytrostruk approved these changes Feb 5, 2026

View reviewed changes

eavanvalkenburg added this pull request to the merge queue Feb 5, 2026

Merged via the queue into microsoft:main with commit 3dc59c8 Feb 5, 2026
41 of 46 checks passed

This was referenced Feb 11, 2026

Python: [Bug]: WorkflowAgent incompatible with add_agent_framework_fastapi_endpoint - streaming interface mismatch #3853

Closed

Python: (ag-ui): fix Workflow.as_agent() streaming regression #3875

Merged

Comments

Conversation

eavanvalkenburg commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation and Context

Description

Contribution Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

markwallace-microsoft commented Jan 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Python Unit Test Overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

eavanvalkenburg commented Feb 2, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

eavanvalkenburg commented Jan 22, 2026 •

edited

Loading

markwallace-microsoft commented Jan 23, 2026 •

edited

Loading