.NET: CI Build time end to end improvement by westey-m · Pull Request #4208 · microsoft/agent-framework

westey-m · 2026-02-24T11:29:10Z

Motivation and Context

Our builds are slow and needs improvement.

Description

Upgrade to XUnit 3 and Microsoft Testing Platform to allow more parallel execution options
Split Build and Test into parallel Jobs, so that Test only needs to build src and tests and not samples
Instead of running tests in a loop with filtering per project, we generate a new filtered slnx upfront, which we can point the test runner at, which runs all tests in parallel.
Builds with integration tests are down from ~25m to ~13m.

Contribution Checklist

The code builds clean without any errors or warnings
The PR follows the Contribution Guidelines
All unit tests pass, and I have added new tests where possible
Is this a breaking change? If yes, add "[BREAKING]" prefix to the title of the PR.

dotnet/eng/scripts/New-FilteredSolution.ps1

Copilot

Pull request overview

Updates the .NET test stack and CI pipeline to reduce end-to-end build time by moving to xUnit v3 on Microsoft Testing Platform and running build/test work in parallel with solution filtering.

Changes:

Migrate .NET tests to xUnit v3 + Microsoft Testing Platform (MTP), including updated async lifetime signatures and skip patterns.
Split CI into parallel build-only and test jobs; generate filtered solutions to avoid building samples for test runs.
Switch coverage collection to Microsoft.Testing.Extensions.CodeCoverage with a shared .runsettings file and updated report globbing.

Reviewed changes

Copilot reviewed 115 out of 115 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
dotnet/tests/coverage.runsettings	Adds MTP code coverage settings (attribute exclusions).
dotnet/tests/OpenAIResponse.IntegrationTests/OpenAIResponseRunTests.cs	Replace attribute-based skipping with runtime skip calls for overridden conformance tests.
dotnet/tests/OpenAIResponse.IntegrationTests/OpenAIResponseRunStreamingTests.cs	Same skip pattern change for streaming conformance tests.
dotnet/tests/OpenAIResponse.IntegrationTests/OpenAIResponseFixture.cs	Update fixture async lifetime to `ValueTask` and add disposal pattern.
dotnet/tests/OpenAIResponse.IntegrationTests/OpenAIResponseChatClientAgentRunTests.cs	Replace `[Fact(Skip=...)]` overrides with runtime skip calls.
dotnet/tests/OpenAIResponse.IntegrationTests/OpenAIResponseChatClientAgentRunStreamingTests.cs	Same as above for streaming.
dotnet/tests/OpenAIChatCompletion.IntegrationTests/OpenAIChatCompletionFixture.cs	Update fixture async lifetime to `ValueTask` + suppress finalization.
dotnet/tests/OpenAIAssistant.IntegrationTests/OpenAIAssistantFixture.cs	Update fixture async lifetime to `ValueTask`; wrap delete task in `ValueTask`.
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/WorkflowTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/TestOutputAdapter.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/PowerFx/WorkflowExpressionEngineTests.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/PowerFx/TemplateExtensionsTests.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/PowerFx/RecalcEngineTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/PowerFx/RecalcEngineFactoryTests.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/ObjectModel/WorkflowActionExecutorTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/ObjectModel/SetVariableExecutorTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/ObjectModel/SetTextVariableExecutorTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/ObjectModel/SetMultipleVariablesExecutorTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/ObjectModel/SendActivityExecutorTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/ObjectModel/RetrieveConversationMessagesExecutorTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/ObjectModel/RetrieveConversationMessageExecutorTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/ObjectModel/ResetVariableExecutorTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/ObjectModel/RequestExternalInputExecutorTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/ObjectModel/QuestionExecutorTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/ObjectModel/ParseValueExecutorTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/ObjectModel/InvokeFunctionToolExecutorTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/ObjectModel/ForeachExecutorTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/ObjectModel/EditTableV2ExecutorTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/ObjectModel/EditTableExecutorTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/ObjectModel/DefaultActionExecutorTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/ObjectModel/CreateConversationExecutorTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/ObjectModel/CopyConversationMessagesExecutorTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/ObjectModel/ConditionGroupExecutorTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/ObjectModel/ClearAllVariablesExecutorTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/ObjectModel/AddConversationMessageExecutorTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/Interpreter/WorkflowModelTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/Events/ExternalInputResponseTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/Events/ExternalInputRequestTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/Events/EventTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/Entities/EntityExtractorTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/Entities/EntityExtractionResultTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/DeclarativeWorkflowTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/DeclarativeWorkflowExceptionTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/CodeGen/WorkflowActionTemplateTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/CodeGen/SetVariableTemplateTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/CodeGen/SetTextVariableTemplateTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/CodeGen/SetMultipleVariablesTemplateTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/CodeGen/RetrieveConversationMessagesTemplateTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/CodeGen/RetrieveConversationMessageTemplateTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/CodeGen/ResetVariableTemplateTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/CodeGen/ProviderTemplateTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/CodeGen/InvokeAzureAgentTemplateTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/CodeGen/GotoTemplateTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/CodeGen/ForeachTemplateTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/CodeGen/EndDialogTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/CodeGen/EndConversationTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/CodeGen/EdgeTemplateTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/CodeGen/DeclarativeEjectionTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/CodeGen/CreateConversationTemplateTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/CodeGen/CopyConversationMessagesTemplateTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/CodeGen/ContinueLoopTemplateTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/CodeGen/ConditionGroupTemplateTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/CodeGen/ClearAllVariablesTemplateTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/CodeGen/BreakLoopTemplateTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/CodeGen/AddConversationMessageTemplateTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests/MediaInputTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests/InvokeFunctionToolWorkflowTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests/FunctionCallingWorkflowTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests/Framework/WorkflowTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests/Framework/TestOutputAdapter.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests/Framework/IntegrationTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests/DeclarativeWorkflowTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests/DeclarativeCodeGenTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests/AzureAgentProviderTest.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.Hosting.AzureFunctions.IntegrationTests/SamplesValidation.cs	Convert async lifetime + add sample build step and `--no-build` runs.
dotnet/tests/Microsoft.Agents.AI.Hosting.AGUI.AspNetCore.IntegrationTests/ToolCallingTests.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.DurableTask.IntegrationTests/TimeToLiveTests.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.DurableTask.IntegrationTests/TestHelper.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.DurableTask.IntegrationTests/OrchestrationTests.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.DurableTask.IntegrationTests/Logging/TestLoggerProvider.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.DurableTask.IntegrationTests/Logging/TestLogger.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.DurableTask.IntegrationTests/ExternalClientTests.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.DurableTask.IntegrationTests/ConsoleAppSamplesValidation.cs	Convert async lifetime + add sample build step and `--no-build` runs.
dotnet/tests/Microsoft.Agents.AI.DurableTask.IntegrationTests/AgentEntityTests.cs	Remove xUnit abstractions import (xUnit v3 migration cleanup).
dotnet/tests/Microsoft.Agents.AI.CosmosNoSql.UnitTests/Microsoft.Agents.AI.CosmosNoSql.UnitTests.csproj	Remove `Xunit.SkippableFact` package reference.
dotnet/tests/Microsoft.Agents.AI.CosmosNoSql.UnitTests/CosmosCheckpointStoreTests.cs	Migrate to ValueTask lifetime + replace skippable facts with runtime skip.
dotnet/tests/Microsoft.Agents.AI.CosmosNoSql.UnitTests/CosmosChatHistoryProviderTests.cs	Migrate to ValueTask lifetime + replace skippable facts with runtime skip.
dotnet/tests/Directory.Build.props	Switch test infra packages to xUnit v3 MTP + MTP coverage extension.
dotnet/tests/CopilotStudio.IntegrationTests/CopilotStudioRunTests.cs	Change skip behavior to runtime skipping and conditional manual verification.
dotnet/tests/CopilotStudio.IntegrationTests/CopilotStudioRunStreamingTests.cs	Same as above for streaming.
dotnet/tests/CopilotStudio.IntegrationTests/CopilotStudioFixture.cs	Make fixture skip when config missing; migrate to ValueTask lifetime.
dotnet/tests/CopilotStudio.IntegrationTests/CopilotStudio.IntegrationTests.csproj	Add warning suppression for updated test runner/tooling.
dotnet/tests/AzureAIAgentsPersistent.IntegrationTests/AzureAIAgentsPersistentStructuredOutputRunTests.cs	Convert skip pattern to runtime skipping for flaky tests.
dotnet/tests/AzureAIAgentsPersistent.IntegrationTests/AzureAIAgentsPersistentFixture.cs	Migrate to ValueTask async lifetime + cleanup.
dotnet/tests/AzureAIAgentsPersistent.IntegrationTests/AzureAIAgentsPersistent.IntegrationTests.csproj	Add warning suppression for updated test runner/tooling.
dotnet/tests/AzureAI.IntegrationTests/AzureAI.IntegrationTests.csproj	Add warning suppression for updated test runner/tooling.
dotnet/tests/AzureAI.IntegrationTests/AIProjectClientFixture.cs	Migrate fixture async lifetime to `ValueTask` and wrap delete task.
dotnet/tests/AzureAI.IntegrationTests/AIProjectClientChatClientAgentRunTests.cs	Convert “skip” to runtime skip call.
dotnet/tests/AzureAI.IntegrationTests/AIProjectClientChatClientAgentRunStreamingTests.cs	Convert “skip” to runtime skip call.
dotnet/tests/AzureAI.IntegrationTests/AIProjectClientAgentStructuredOutputRunTests.cs	Convert “skip” to runtime skip call + update structured-output fixture init to ValueTask.
dotnet/tests/AzureAI.IntegrationTests/AIProjectClientAgentRunTests.cs	Convert “skip” to runtime skip call(s).
dotnet/tests/AzureAI.IntegrationTests/AIProjectClientAgentRunStreamingTests.cs	Convert “skip” to runtime skip call(s).
dotnet/tests/AnthropicChatCompletion.IntegrationTests/AnthropicSkillsIntegrationTests.cs	Move config load inside tests and runtime-skip local-only tests.
dotnet/tests/AnthropicChatCompletion.IntegrationTests/AnthropicChatCompletionRunTests.cs	Remove “skip-all” wrapper base and rely on fixture/test runtime skipping.
dotnet/tests/AnthropicChatCompletion.IntegrationTests/AnthropicChatCompletionRunStreamingTests.cs	Same as above for streaming.
dotnet/tests/AnthropicChatCompletion.IntegrationTests/AnthropicChatCompletionFixture.cs	Runtime-skip local-only tests; migrate async lifetime to ValueTask.
dotnet/tests/AnthropicChatCompletion.IntegrationTests/AnthropicChatCompletionChatClientAgentRunTests.cs	Remove “skip-all” wrapper base and rely on runtime skipping.
dotnet/tests/AnthropicChatCompletion.IntegrationTests/AnthropicChatCompletionChatClientAgentRunStreamingTests.cs	Same as above for streaming.
dotnet/tests/AnthropicChatCompletion.IntegrationTests/AnthropicChatCompletion.IntegrationTests.csproj	Add warning suppression for updated test runner/tooling.
dotnet/tests/AgentConformance.IntegrationTests/AgentTests.cs	Update base test fixture lifecycle to xUnit v3 `ValueTask` pattern.
dotnet/global.json	Configure Microsoft Testing Platform runner for `dotnet test`.
dotnet/Directory.Packages.props	Update/replace test-related package versions for xUnit v3 + MTP + coverage extension.
dotnet/.github/skills/build-and-test/SKILL.md	Update local dev instructions for MTP CLI syntax and coverage usage.
.github/workflows/dotnet-build-and-test.yml	Split build/test into parallel jobs; add filtered-solution generation and MTP coverage/test args.
.github/workflows/New-FilteredSolution.ps1	New script to generate framework/test-type filtered `.slnx` solutions for CI.

Comments suppressed due to low confidence (4)

dotnet/tests/Microsoft.Agents.AI.Hosting.AzureFunctions.IntegrationTests/SamplesValidation.cs:51

SamplesValidation implements IAsyncLifetime, but the dispose method is implemented as IAsyncDisposable.DisposeAsync(). If IAsyncLifetime expects DisposeAsync() (as other tests in this PR do), this won’t satisfy the interface and the cleanup hook won’t run. Implement ValueTask IAsyncLifetime.DisposeAsync() (or a public ValueTask DisposeAsync()), and remove the IAsyncDisposable explicit implementation unless the type explicitly implements IAsyncDisposable too.
dotnet/tests/Microsoft.Agents.AI.DurableTask.IntegrationTests/ConsoleAppSamplesValidation.cs:45
ConsoleAppSamplesValidation implements IAsyncLifetime, but the dispose method is implemented as IAsyncDisposable.DisposeAsync(). This likely doesn’t fulfill IAsyncLifetime.DisposeAsync() and can prevent xUnit from calling the fixture teardown. Implement ValueTask IAsyncLifetime.DisposeAsync() (or a public ValueTask DisposeAsync()), consistent with other updated fixtures in this PR.
dotnet/tests/Microsoft.Agents.AI.CosmosNoSql.UnitTests/CosmosCheckpointStoreTests.cs:66
InitializeAsync calls SkipIfEmulatorNotAvailable() before attempting the connection check that sets _emulatorAvailable. As written, local runs will always skip unless COSMOS_EMULATOR_AVAILABLE=true is set, regardless of whether the emulator is actually reachable. Consider moving the skip decision after the connection attempt (or make SkipIfEmulatorNotAvailable only consult the env var during CI).

    public async ValueTask InitializeAsync()
    {
        // Fail fast if emulator is not available
        this.SkipIfEmulatorNotAvailable();

        // Check environment variable to determine if we should preserve containers
        // Set COSMOS_PRESERVE_CONTAINERS=true to keep containers and data for inspection
        this._preserveContainer = string.Equals(Environment.GetEnvironmentVariable("COSMOS_PRESERVE_CONTAINERS"), "true", StringComparison.OrdinalIgnoreCase);

dotnet/tests/Microsoft.Agents.AI.CosmosNoSql.UnitTests/CosmosChatHistoryProviderTests.cs:71

InitializeAsync calls SkipIfEmulatorNotAvailable() before the connection attempt that sets _emulatorAvailable, so the “Locally: skip if connection check failed” path can’t work. This causes local runs to always skip unless COSMOS_EMULATOR_AVAILABLE=true is set. Move the skip check after the connection attempt, or adjust the skip logic so local availability is determined by the try/catch connection test.

    public async ValueTask InitializeAsync()
    {
        // Fail fast if emulator is not available
        this.SkipIfEmulatorNotAvailable();

        // Check environment variable to determine if we should preserve containers
        // Set COSMOS_PRESERVE_CONTAINERS=true to keep containers and data for inspection
        this._preserveContainer = string.Equals(Environment.GetEnvironmentVariable("COSMOS_PRESERVE_CONTAINERS"), "true", StringComparison.OrdinalIgnoreCase);

        this._connectionString = $"AccountEndpoint={s_emulatorEndpoint};AccountKey={s_emulatorKey}";

dotnet/tests/Microsoft.Agents.AI.Hosting.AzureFunctions.IntegrationTests/SamplesValidation.cs

dotnet/tests/Microsoft.Agents.AI.DurableTask.IntegrationTests/ConsoleAppSamplesValidation.cs

rogerbarreto · 2026-02-25T14:17:07Z

I see double checks running for both `push` and `pull_request` should we have just the `pull_request` running in this context ?

rogerbarreto · 2026-02-25T14:20:29Z

CAn we generate the build/solution artifact and use that in the tests later se we can skip build during tests and use the actual built artifact straight away, gaining even more time back?

westey-m · 2026-02-25T14:24:54Z

Can we generate the build/solution artifact and use that in the tests later se we can skip build during tests and use the actual built artifact straight away, gaining even more time back?

This would require having tests depend on building everything, which is what we were doing before. Building everything takes up to 9 mins, while building just src and tests take around 2 mins. Its therefore faster to just build src/tests (2mins) and then run the tests (6/7mins), rather than building everything (9mins) and then running the tests (6/7mins). The build now builds everything in parallel to building and running just tests.

westey-m · 2026-02-25T15:16:08Z

I see double checks running for both push and pull_request should we have just the pull_request running in this context ?

This is unrelated existing behavior and is because this is a feature branch. See here

* .NET: Add Microsoft Fabric sample #3674 (#4230) Co-authored-by: Chris <66376200+crickman@users.noreply.github.com> * Python: Phase 2: Embedding clients for Ollama, Bedrock, and Azure AI Inference (#4207) * Phase 2: Embedding clients for Ollama, Bedrock, and Azure AI Inference Add embedding client implementations to existing provider packages: - OllamaEmbeddingClient: Text embeddings via Ollama's embed API - BedrockEmbeddingClient: Text embeddings via Amazon Titan on Bedrock - AzureAIInferenceEmbeddingClient: Text and image embeddings via Azure AI Inference, supporting Content | str input with separate model IDs for text (AZURE_AI_INFERENCE_EMBEDDING_MODEL_ID) and image (AZURE_AI_INFERENCE_IMAGE_EMBEDDING_MODEL_ID) endpoints Additional changes: - Rename EmbeddingCoT -> EmbeddingT, EmbeddingOptionsCoT -> EmbeddingOptionsT - Add otel_provider_name passthrough to all embedding clients - Register integration pytest marker in all packages - Add lazy-loading namespace exports for Ollama and Bedrock embeddings - Add image embedding sample using Cohere-embed-v3-english - Add azure-ai-inference dependency to azure-ai package Part of #1188 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix mypy duplicate name and ruff lint issues - Rename second 'vector' variable to 'img_vector' in image embedding loop - Combine nested with statements in tests - Remove unused result assignments in tests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * updates from feedback * Fix CI failures in embedding usage handling - Fix Azure AI embedding mypy issues by normalizing vectors to list[float], safely accumulating optional usage token fields, and filtering None entries before constructing GeneratedEmbeddings - Avoid Bandit false positive by initializing usage details as an empty dict - Update OpenAI embedding tests to assert canonical usage keys (input_token_count/total_token_count) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * [Purview] Mark responses as responses and fix epoch bug for python long overflow (#4225) * .NET: Support InvokeMcpTool for declarative workflows (#4204) * Initial implementation of InvokeMcpTool in declarative workflow * Cleaned up sample implementation * Updated sample comments. * Added missing executor routing attribute * Fix PR comments. * Updated based on PR comments. * Updated based on PR comments. * Removed unnecessary using statement. * Update Python package versions to rc2 (#4258) - Bump core and azure-ai to 1.0.0rc2 - Bump preview packages to 1.0.0b260225 - Update dependencies to >=1.0.0rc2 - Add CHANGELOG entries for changes since rc1 - Update uv.lock Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * .NET: Fixing issue where OpenTelemetry span is never exported in .NET in-process workflow execution (#4196) * 1. Add reproduction test for issue #4155: workflow.run Activity never stopped in streaming OffThread path The WorkflowRunActivity_IsStopped_Streaming_OffThread test demonstrates that the workflow.run OpenTelemetry Activity created in StreamingRunEventStream.RunLoopAsync is started but never stopped when using the OffThread/Default streaming execution. The background run loop keeps running after event consumption completes, so the using Activity? declaration never disposes until explicit StopAsync() is called. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> 2. Fix workflow.run Activity never stopped in streaming OffThread execution (#4155) The workflow.run OpenTelemetry Activity in StreamingRunEventStream.RunLoopAsync was scoped to the method lifetime via 'using'. Since the run loop only exits on cancellation, the Activity was never stopped/exported until explicit disposal. Fix: Remove 'using' and explicitly dispose the Activity when the workflow reaches Idle status (all supersteps complete). A safety-net disposal in the finally block handles cancellation and error paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add root-level workflow.session activity spanning run loop lifetime\n\nImplements two-level telemetry hierarchy per PR feedback from lokitoth:\n- workflow.session: spans the entire run loop / stream lifetime\n- workflow_invoke: per input-to-halt cycle, nested within the session\n\nThis ensures the session activity stays open across multiple turns,\nwhile individual run activities are created and disposed per cycle.\n\nAlso fixes linkedSource CancellationTokenSource disposal leak in\nStreamingRunEventStream (added using declaration)." * Address Copilot review: fix Activity/CTS disposal, rename activity, add error tag\n\n1. LockstepRunEventStream: Remove 'using' from Activity in async iterator\n and manually dispose in finally block (fixes #4155 pattern). Also dispose\n linkedSource CTS in finally to prevent leak.\n2. Tags.cs: Add ErrorMessage (\"error.message\") tag for runtime errors,\n distinct from BuildErrorMessage (\"build.error.message\").\n3. ActivityNames: Rename WorkflowRun from \"workflow_invoke\" to \"workflow.run\"\n for cross-language consistency.\n4. WorkflowTelemetryContext: Fix XML doc to say \"outer/parent span\" instead\n of \"root-level span\".\n5. ObservabilityTests: Assert WorkflowSession absence when DisableWorkflowRun\n is true.\n6. WorkflowRunActivityStopTests: Fix streaming test race by disposing\n StreamingRun before asserting activities are stopped.\n7. StreamingRunEventStream/LockstepRunEventStream: Use Tags.ErrorMessage\n instead of Tags.BuildErrorMessage for runtime error events." * Review fixes: revert workflow_invoke rename, use 'using' for linkedSource, move SessionStarted earlier\n\n- Revert ActivityNames.WorkflowRun back to \"workflow_invoke\" (OTEL semantic convention contract)\n- Use 'using' declaration for linkedSource CTS in LockstepRunEventStream (no timing sensitivity)\n- Move SessionStarted event before WaitForInputAsync in StreamingRunEventStream to match Lockstep behavior" * Improve naming and comments in WorkflowRunActivityStopTests" * Prevent session Activity.Current leak in lockstep mode, add nesting test Save and restore Activity.Current in LockstepRunEventStream.Start() so the session activity doesn't leak into caller code via AsyncLocal. Re-establish Activity.Current = sessionActivity before creating the run activity in TakeEventStreamAsync to preserve parent-child nesting. Add test verifying app activities after RunAsync are not parented under the session, and that the workflow_invoke activity nests under the session." * Fix stale XML doc: WorkflowRun -> WorkflowInvoke in ObservabilityTests --------- Co-authored-by: alliscode <bentho@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Python / .NET Samples - Restructure and Improve Samples (Feature Branc… (#4092) * Python: .NET Samples - Restructure and Improve Samples (Feature Branch) (#4091) * Moved by agent (#4094) * Fix readme links * .NET Samples - Create `04-hosting` learning path step (#4098) * Agent move * Agent reorderd * Remove A2A section from README Removed A2A section from the Getting Started README. * Agent fixed links * Fix broken sample links in durable-agents README (#4101) * Initial plan * Fix broken internal links in documentation Co-authored-by: crickman <66376200+crickman@users.noreply.github.com> * Revert template link changes; keep only durable-agents README fix Co-authored-by: crickman <66376200+crickman@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: crickman <66376200+crickman@users.noreply.github.com> * .NET Samples - Create `03-workflows` learning path step (#4102) * Fix solution project path * Python: Fix broken markdown links to repo resources (outside /docs) (#4105) * Initial plan * Fix broken markdown links to repo resources Co-authored-by: crickman <66376200+crickman@users.noreply.github.com> * Update README to rename .NET Workflows Samples section --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: crickman <66376200+crickman@users.noreply.github.com> * .NET Samples - Create `02-agents` learning path step (#4107) * .NET: Fix broken relative link in GroupChatToolApproval README (#4108) * Initial plan * Fix broken link in GroupChatToolApproval README Co-authored-by: crickman <66376200+crickman@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: crickman <66376200+crickman@users.noreply.github.com> * Update labeler configuration for workflow samples * .NET - Reorder Agents samples to start from Step01 instead of Step04 (#4110) * Fix solution * Resolve new sample paths * Move new AgentSkills and AgentWithMemory_Step04 samples * Fix link * Fix readme path * fix: update stale dotnet/samples/Durable path reference in AGENTS.md Co-authored-by: crickman <66376200+crickman@users.noreply.github.com> * Moved new sample * Update solution * Resolve merge (new sample) * Sync to new sample - FoundryAgents_Step21_BingCustomSearch * Updated README * .NET Samples - Configuration Naming Update (#4149) * .NET: Restore AzureFunctions index parity with ConsoleApps under DurableAgents samples (#4221) * Clean-up `05_host_your_agent` * Config setting consistency * Refine samples * AGENTS.md * Move new samples * Re-order samples * Move new project and fixup solution * Fixup model config * Fix up new UT project --------- Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> * Python: Fix Bedrock embedding test stub missing meta attribute (#4287) * Fix Bedrock embedding test stub missing meta attribute * Increase test coverage so gate passes * Python: (ag-ui): fix approval payloads being re-processed on subsequent conversation turns (#4232) * Fix ag-ui tool call issue * Safe json fix * Python: Update workflow orchestration samples to use AzureOpenAIResponsesClient (#4285) * Update workflow orchestration samples to use AzureOpenAIResponsesClient * Fix broken link * Move scripts to scripts folder --------- Co-authored-by: Roger Barreto <19890735+rogerbarreto@users.noreply.github.com> Co-authored-by: Chris <66376200+crickman@users.noreply.github.com> Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Rishabh Chawla <rishabhchawla1995@gmail.com> Co-authored-by: Peter Ibekwe <109177538+peibekwe@users.noreply.github.com> Co-authored-by: Dmytro Struk <13853051+dmytrostruk@users.noreply.github.com> Co-authored-by: Ben Thomas <ben.thomas@microsoft.com> Co-authored-by: alliscode <bentho@microsoft.com> Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Co-authored-by: Evan Mattson <35585003+moonbox3@users.noreply.github.com>

* Disable flakey OffThread test * Disable additional OffThread test * Disable a further test * Disable all observability tests

.NET: Upgrade to XUnit 3 and Microsoft Testing Platform (#4176)

6c3b9d4

markwallace-microsoft added documentation Improvements or additions to documentation .NET labels Feb 24, 2026

westey-m temporarily deployed to integration February 24, 2026 11:29 — with GitHub Actions Inactive

Fix copilot studio integration tests failure (#4209)

d04228d

westey-m temporarily deployed to integration February 24, 2026 12:07 — with GitHub Actions Inactive

westey-m had a problem deploying to integration February 24, 2026 12:07 — with GitHub Actions Failure

Fix anthropic integration tests and skip reason (#4211)

fb73231

westey-m temporarily deployed to integration February 24, 2026 14:48 — with GitHub Actions Inactive

westey-m had a problem deploying to integration February 24, 2026 14:48 — with GitHub Actions Failure

Remove accidental add of code coverage for integration tests (#4219)

be5ed93

westey-m temporarily deployed to integration February 24, 2026 17:49 — with GitHub Actions Inactive

Add solution filtered parallel test run (#4226)

a39e324

westey-m had a problem deploying to integration February 24, 2026 21:22 — with GitHub Actions Failure

Fix build paths (#4228)

f1585b1

westey-m had a problem deploying to integration February 24, 2026 21:46 — with GitHub Actions Failure

westey-m temporarily deployed to integration February 24, 2026 21:46 — with GitHub Actions Inactive

westey-m temporarily deployed to integration February 25, 2026 13:50 — with GitHub Actions Inactive

westey-m marked this pull request as ready for review February 25, 2026 14:05

Copilot AI review requested due to automatic review settings February 25, 2026 14:05

Copilot started reviewing on behalf of westey-m February 25, 2026 14:05 View session

rogerbarreto reviewed Feb 25, 2026

View reviewed changes

dotnet/eng/scripts/New-FilteredSolution.ps1 Show resolved Hide resolved

Copilot AI reviewed Feb 25, 2026

View reviewed changes

dotnet/tests/Microsoft.Agents.AI.Hosting.AzureFunctions.IntegrationTests/SamplesValidation.cs Show resolved Hide resolved

dotnet/tests/Microsoft.Agents.AI.DurableTask.IntegrationTests/ConsoleAppSamplesValidation.cs Show resolved Hide resolved

Address PR comments (#4255)

d4f9579

westey-m temporarily deployed to integration February 25, 2026 14:54 — with GitHub Actions Inactive

westey-m temporarily deployed to integration February 25, 2026 14:55 — with GitHub Actions Inactive

lokitoth approved these changes Feb 25, 2026

View reviewed changes

SergeyMenshykh approved these changes Feb 25, 2026

View reviewed changes

westey-m and others added 9 commits February 25, 2026 17:25

Merge branch 'main' into feature-xunit3-mtp-upgrade

cc1ef73

Merge branch 'main' into feature-xunit3-mtp-upgrade

204d01c

Fix encoding (#4309)

6bc5f54

Disable Parallelization for WorkflowRunActivityStopTests (#4313)

279386d

Revert parallel disable (#4324)

7cb4e78

Merge branch 'main' into feature-xunit3-mtp-upgrade

5dcf45b

.NET: Disable flakey Workflow Observability tests (#4416)

21e7873

* Disable flakey OffThread test * Disable additional OffThread test * Disable a further test * Disable all observability tests

Merge branch 'main' into feature-xunit3-mtp-upgrade

213fcb9

SergeyMenshykh approved these changes Mar 3, 2026

View reviewed changes

westey-m added 4 commits March 4, 2026 14:28

Merge branch 'main' into feature-xunit3-mtp-upgrade

e730fc1

Merge branch 'main' into feature-xunit3-mtp-upgrade

5441fec

Merge branch 'main' into feature-xunit3-mtp-upgrade

9203cbe

Merge branch 'main' into feature-xunit3-mtp-upgrade

410b68e

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

.NET: CI Build time end to end improvement#4208

.NET: CI Build time end to end improvement#4208
westey-m wants to merge 29 commits intomainfrom
feature-xunit3-mtp-upgrade

westey-m commented Feb 24, 2026 •

edited

Loading

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

rogerbarreto commented Feb 25, 2026

Uh oh!

rogerbarreto commented Feb 25, 2026

Uh oh!

westey-m commented Feb 25, 2026

Uh oh!

westey-m commented Feb 25, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

westey-m commented Feb 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation and Context

Description

Contribution Checklist

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

rogerbarreto commented Feb 25, 2026

Uh oh!

rogerbarreto commented Feb 25, 2026

Uh oh!

westey-m commented Feb 25, 2026

Uh oh!

westey-m commented Feb 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

westey-m commented Feb 24, 2026 •

edited

Loading

westey-m commented Feb 25, 2026 •

edited

Loading