Python: (ag-ui): Add Workflow Support, Harden Streaming Semantics, and add Dynamic Handoff Demo

[moonbox3]

Motivation and Context

• Added first-class AG-UI workflow execution via AgentFrameworkWorkflow, with endpoint support for both wrapped agents and workflows using .run(...).
• Removed duplicated emit logic from _agent_run and consolidated shared emission behavior in _run_common.
• Fixed ToolCallResultEvent serialization bug for non-string tool results.
• Hardened error handling:
  • endpoint now returns proper HTTP error responses (no silent 200 with error dict),
  • streaming/generator/encoding failures now emit RUN_ERROR instead of silently terminating.
• Added usage propagation as AG-UI CUSTOM (name="usage") and surfaced diagnostics in the handoff demo UI.
• Added multimodal capabilities and added agent/workflow multimodal coverage.
• Added handoff demo to show dynamic routing (.add_handoff(...)) and added HITL-gated replacement approval (submit_replacement, submit_refund) as examples on HITL tool approvals with AG-UI.
• Handles duplicate final assistant text replay (chunk + final snapshot tail duplication) with backend and demo UI dedupe guards.

Breaking Changes

• Removed run_agent(...); now uses run(...) for both an agent and a workflow run.
• Narrowed agent_framework.ag_ui facade exports to a more stable public surface.

Demo Screenshots

Description

• Closes Python: [Feature Request] Support custom WorkflowEvent emission for AG-UI/CopilotKit integration #3872
• Closes Python: [Feature]: [Python]: Timeline and Plans for First-Class UI Events Support in AG-UI #3632
• Closes Python: Does AG-UI Support Image Inputs for Agents? #3514
• Addresses Is it possible to run a multi-agent workflow with AG-UI? #3227

Contribution Checklist

• The code builds clean without any errors or warnings
• The PR follows the Contribution Guidelines
• All unit tests pass, and I have added new tests where possible
• Is this a breaking change? If yes, add "[BREAKING]" prefix to the title of the PR.

[markwallace-microsoft]

Python Test Coverage Report •

File	Stmts	Miss	Cover	Missing
packages/ag-ui/agent_framework_ag_ui
_agent.py	36	1	97%	62
_agent_run.py	405	82	79%	147–154, 193–194, 201, 298, 310, 314, 316–317, 333, 360–361, 416–418, 430–432, 586, 588–589, 642, 653–654, 661, 710–712, 729, 735, 743, 745, 753, 783–789, 792–795, 797–806, 809, 817–820, 827, 830–831, 836, 842–844, 848, 853–857, 883–885
_client.py	147	17	88%	85–86, 90–94, 98–102, 265, 292, 463–465
_endpoint.py	72	12	83%	68, 104, 108–110, 114–118, 136–137
_event_converters.py	80	0	100%
_http_service.py	46	0	100%
_message_adapters.py	590	129	78%	92, 102–103, 112–115, 118–122, 124–129, 132, 141–147, 150, 154–156, 165–167, 187, 193–195, 225, 238–239, 249–250, 290–291, 293–295, 302, 310, 323–324, 326, 338–339, 341–342, 357–359, 361, 366, 400–403, 405–407, 414, 416, 422–424, 426–429, 438–439, 450, 454–456, 463, 466, 510, 513, 515, 518, 521, 537, 554, 576, 607, 612, 623–624, 672, 688–689, 755–758, 760, 766, 774–775, 777, 781–784, 797, 852, 886–889, 891, 956, 994, 997, 999, 1002, 1004, 1010–1011, 1013
_run_common.py	194	31	84%	46, 51–52, 54, 61, 65, 69–70, 72, 96, 173, 191–192, 204–205, 219, 246–248, 273–274, 280–285, 372, 374, 377–378
_types.py	42	0	100%
_workflow.py	44	1	97%	72
_workflow_run.py	502	121	75%	72, 82, 89, 112, 133–134, 154, 166, 173, 177, 181, 185–186, 192, 194, 202–206, 212, 218, 233, 238, 241, 257, 261, 266, 269, 276, 279–287, 291–293, 295, 304–305, 308, 319, 348–349, 353, 358, 370, 372, 375, 386, 397, 404, 408, 419, 421, 431, 437–439, 441–445, 447–450, 454–462, 470, 477–478, 483, 487–493, 497–503, 515, 543, 556–558, 585–586, 589–593, 600, 632, 641, 657, 686–688, 700, 722
packages/ag-ui/agent_framework_ag_ui/_orchestration
_helpers.py	108	0	100%
packages/core/agent_framework
_tools.py	856	92	89%	166–167, 307, 309, 327–329, 336, 354, 368, 375, 382, 398, 400, 407, 444, 469, 473, 490–492, 539–541, 563, 617, 639, 702–708, 744, 755–766, 788–790, 795, 799, 813–815, 854, 923, 933, 943, 999, 1030, 1049, 1311, 1368, 1388, 1459–1463, 1585, 1589, 1613, 1639, 1641, 1657, 1659, 1744, 1774, 1794, 1796, 1847, 1910, 2094–2095, 2132, 2145, 2155–2156, 2191–2192, 2252
packages/orchestrations/agent_framework_orchestrations
_handoff.py	387	59	84%	105–106, 108, 163–173, 175, 177, 179, 184, 281, 287, 319, 342, 364, 420, 445, 503, 535, 592–593, 625, 633, 637–638, 676–678, 683–685, 801, 804, 811, 816, 878, 883, 890, 900, 902, 921, 923, 1005–1006, 1038–1039, 1121, 1128, 1200–1201, 1203
_orchestrator_helpers.py	15	2	86%	66–67
TOTAL	22131	3472	84%

Python Unit Test Overview

Tests	Skipped	Failures	Errors	Time
4254	240 💤	0 ❌	0 🔥	1m 14s ⏱️

[Copilot]

Pull request overview

This pull request adds first-class AG-UI workflow execution support, hardens streaming error handling, and introduces a dynamic handoff demo. The changes enable workflows to be served directly through AG-UI endpoints alongside agents, add multimodal input support, and improve error handling to emit RUN_ERROR events instead of silently terminating streams.

Changes:

• Introduces AgentFrameworkWorkflow wrapper for native Workflow objects with thread-scoped factory support
• Breaking change: renames run_agent() to run() across all AG-UI protocol implementations
• Adds streaming error handling that emits RUN_ERROR events for stream/encoding failures
• Implements multimodal input parsing for image/audio/video/document content types
• Adds comprehensive handoff workflow demo with tool approval gates

Reviewed changes

Copilot reviewed 53 out of 56 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
python/packages/ag-ui/agent_framework_ag_ui/_workflow.py	New workflow wrapper with thread-scoped factory support
python/packages/ag-ui/agent_framework_ag_ui/_workflow_run.py	Native workflow-to-AG-UI event stream mapper
python/packages/ag-ui/agent_framework_ag_ui/_run_common.py	Shared emission helpers extracted from agent/workflow runners
python/packages/ag-ui/agent_framework_ag_ui/_agent.py	Breaking change: run_agent() → run()
python/packages/ag-ui/agent_framework_ag_ui/_endpoint.py	Error handling improvements and workflow protocol support
python/packages/ag-ui/agent_framework_ag_ui/_message_adapters.py	Multimodal input parsing with base64 decoding
python/packages/orchestrations/agent_framework_orchestrations/_handoff.py	JSON serialization fix for handoff function results
python/samples/demos/ag_ui_workflow_handoff/backend/server.py	New handoff demo backend
python/samples/demos/ag_ui_workflow_handoff/frontend/	React frontend for handoff demo

Files not reviewed (1)

• python/samples/demos/ag_ui_workflow_handoff/frontend/package-lock.json: Language not supported