Add HostedToolSearchTool and SearchableAIFunctionDeclaration for tool search / deferred loading support

[Copilot]

Implements tool search and deferred loading support (issue #7371): a HostedToolSearchTool marker and SearchableAIFunctionDeclaration decorator that signals per-tool deferred loading to supporting providers. OpenAI Responses API support included; Anthropic follows separately.

New abstractions (Microsoft.Extensions.AI.Abstractions)

• HostedToolSearchTool — marker AITool (same pattern as HostedWebSearchTool/HostedCodeInterpreterTool); maps to the tool_search hosted tool.
• SearchableAIFunctionDeclaration — decorator wrapping AIFunctionDeclaration that signals deferred loading to supporting providers. Includes an optional Namespace property for organizing tools into namespaces. Detected via tool.GetService<SearchableAIFunctionDeclaration>().
• DelegatingAIFunctionDeclaration — now public (was internal); base class for SearchableAIFunctionDeclaration that passes through calls to an inner AIFunctionDeclaration.
• HostedMcpServerTool.DeferLoadingTools — bool property signaling deferred loading for MCP server tools when tool search is enabled.

All new types are stable (not marked [Experimental]).

OpenAI provider (Microsoft.Extensions.AI.OpenAI)

• HostedToolSearchTool → deserialized ResponseTool from {"type":"tool_search"} via AOT-safe ModelReaderWriter.Read with OpenAIContext.Default (created per-call, not cached). Includes a comment referencing openai/openai-dotnet#1053 as a temporary workaround.
• Deferred loading: provider detects SearchableAIFunctionDeclaration via tool.GetService<SearchableAIFunctionDeclaration>() and patches defer_loading: true onto matching FunctionTool instances
• Namespace grouping: searchable functions with a Namespace are grouped into namespace ResponseTool arrays
• Plain (non-searchable) AIFunctionDeclaration tools are sent without deferred loading
• HostedMcpServerTool with DeferLoadingTools = true gets defer_loading patched via Patch

Tests

• Unit tests (OpenAIResponseClientTests.cs): VerbatimHttpHandler-based tests validating the exact JSON request body sent for various configurations: tool_search only, searchable functions with deferred loading, mixed searchable and plain functions, and mixed with other hosted tools (web search)
• Conversion tests (OpenAIConversionTests.cs): Tests for AsOpenAIResponseTool extension covering tool_search conversion, defer_loading patching via SearchableAIFunctionDeclaration, and plain function without deferred loading
• Abstractions tests (HostedToolSearchToolTests.cs, SearchableAIFunctionDeclarationTests.cs): Tests for tool properties, delegation, GetService discovery, and CreateToolSet
• Integration test (OpenAIResponseClientIntegrationTests.cs): UseToolSearch_WithDeferredFunctions test exercising HostedToolSearchTool with SearchableAIFunctionDeclaration-wrapped functions against the OpenAI API

Usage

// Wrap specific functions as searchable (deferred loading)
var options = new ChatOptions
{
    Tools =
    [
        new HostedToolSearchTool(),
        new SearchableAIFunctionDeclaration(getWeather),
        new SearchableAIFunctionDeclaration(getForecast),
        importantTool, // not wrapped — sent without defer_loading
    ]
};

// Use CreateToolSet helper to wrap all functions at once, optionally with a namespace
var tools = SearchableAIFunctionDeclaration.CreateToolSet(
    [getWeather, getForecast, getNews],
    @namespace: "weather_tools");

var response = await chatClient.GetResponseAsync(messages, new() { Tools = tools });

Original prompt

Problem

Implement tool search and deferred loading support as described in #7371. Both OpenAI and Anthropic now support tool search, where tool definitions can be sent with deferred loading (only name/description sent upfront, full schema deferred) and a special tool_search hosted tool is included that the model can invoke to search for and load full tool definitions on demand.

Design

Follow Option A from the issue discussion — a HostedToolSearchTool marker tool + a SearchableAIFunctionDeclaration decorator, consistent with existing patterns (HostedWebSearchTool, ApprovalRequiredAIFunction, etc.).

Requirements

1. New types in Microsoft.Extensions.AI.Abstractions

HostedToolSearchTool (in src/Libraries/Microsoft.Extensions.AI.Abstractions/Tools/HostedToolSearchTool.cs)

• A new AITool subclass, following the exact same pattern as HostedWebSearchTool and HostedCodeInterpreterTool.
• Name returns "tool_search".
• Has the same constructor pattern (default + one taking IReadOnlyDictionary<string, object?>? additionalProperties).
• Must be marked [Experimental(DiagnosticIds.Experiments.AIToolSearch, UrlFormat = DiagnosticIds.UrlFormat)].

SearchableAIFunctionDeclaration (in src/Libraries/Microsoft.Extensions.AI.Abstractions/Functions/SearchableAIFunctionDeclaration.cs)

• A new class that extends DelegatingAIFunctionDeclaration (which is currently internal). Important: DelegatingAIFunctionDeclaration is the declaration-only delegating base (not DelegatingAIFunction which requires AIFunction). This is because SearchableAIFunctionDeclaration should work with AIFunctionDeclaration instances that may not have InvokeAsync.
• Constructor takes AIFunctionDeclaration innerFunction and string? namespaceName = null.
• Has a Namespace property (string?) for grouping related tools.
• Must be sealed.
• Must be marked [Experimental(DiagnosticIds.Experiments.AIToolSearch, UrlFormat = DiagnosticIds.UrlFormat)].
• Include a static convenience helper method: public static IList<AITool> CreateToolSet(IEnumerable<AIFunctionDeclaration> functions, string? namespaceName = null, IReadOnlyDictionary<string, object?>? toolSearchProperties = null) that wraps all functions as SearchableAIFunctionDeclaration and prepends a HostedToolSearchTool, returning a complete tool list ready for ChatOptions.Tools.

2. DiagnosticIds update

In src/Shared/DiagnosticIds/DiagnosticIds.cs, add a new constant in the Experiments class:

internal const string AIToolSearch = AIExperiments;

Place it alongside the other AI experiment constants (near AIWebSearch, AICodeInterpreter, etc.).

3. OpenAI provider implementation

In src/Libraries/Microsoft.Extensions.AI.OpenAI/OpenAIResponsesChatClient.cs:

In ToResponseTool(AITool tool, ChatOptions? options = null) method:

Add handling for HostedToolSearchTool — this maps to the OpenAI tool_search response tool. Since the underlying OpenAI .NET SDK likely doesn't have a ToolSearchTool class yet, you need to manually construct a ResponseTool from JSON. Cache the deserialized ResponseTool instance in a static field so it's only created once. Use the ModelReaderWriter pattern or direct JSON deserialization to create a ResponseTool from the JSON {"type": "tool_search"}. Pattern:

case HostedToolSearchTool:
    return s_toolSearchResponseTool ??= ModelReaderWriter.Read<ResponseTool>(BinaryData.FromString("""{"type": "tool_search"}"""))!;

Add a private static ResponseTool? s_toolSearchResponseTool; field to cache it.

For SearchableAIFunctionDeclaration: When an AIFunctionDeclaration is detected as having a SearchableAIFunctionDeclaration via GetService<SearchableAIFunctionDeclaration>(), the generated FunctionTool should have defer_loading set to true and optionally include the namespace metadata. Since the OpenAI SDK's FunctionTool class may not have these properties yet, use the Patch property to set them on the JSON. The check should happen in the existing case AIFunctionDeclaration aiFunction: branch — after calling ToResponseTool(aiFunction, options), check if the original tool (or aiFunction) has GetService<SearchableAIFunctionDeclaration>() and if so, patch the resulting FunctionTool with defer_loading and namespace. This is done in the ToResponseTool(AITool, ChatOptions?) method so it doesn't infect the general ToResponseTool(AIFunctionDeclaration, ChatOptions?) helper. Specifically, the case AIFunctionDeclaration aiFunction: case should become:

case AIFunctionDeclaration aiFunction:
    var functionTool = ToResponseTool(aiFunction, options);
    if (tool.GetService<SearchableAIFunctionDeclaration>() is { } searchable)
    {
        functionTool.Patch.Set("$.defer_loading"u8, "true"u8);
        if (searchable.Namespace is { } ns)
        {
   ...

</details>



<!-- START COPILOT CODING AGENT SUFFIX -->

*This pull request was created from Copilot chat.*
>

<!-- START COPILOT CODING AGENT TIPS -->
---

✨ Let Copilot coding agent [set things up for you](https://github.com/dotnet/extensions/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot) — coding agent works faster and does higher quality work when set up for your repo.

 ###### Microsoft Reviewers: [Open in CodeFlow](https://microsoft.github.io/open-pr/?codeflow=https://github.com/dotnet/extensions/pull/7377)

[stephentoub]

@copilot, I'm wondering if a different design would be more beneficial. What would you think about isolating it to just the HostedToolSearchTool, which would have two collections of either AITool instances or string names, one for Enable and one for Disable. By default, both of these collections would be null, in which case it would treat every tool as Enable and mark every tool as defer-loading. If Disable was non-null but Enable was null, then it would treat every tool as Enable except for those listed in Disable. Conversely if Enable was non-null and Disable was null, it would only treat those tools in Enable as defer-loading, and if both were non-null, it would treat everything in Enable as defer-loading unless it was also listed in Disable. Thoughts? Is the dominant case for tool search with OpenAI and Anthropic one where most tools are in the same boat and only zero to a small number are special-cased the other way? Or is the right way to model this what we currently have?

[Copilot]

@copilot, I'm wondering if a different design would be more beneficial. What would you think about isolating it to just the HostedToolSearchTool, which would have two collections of ei...

Yes, I think the dominant case is indeed "most/all tools are searchable" — the whole point of tool search is to scale to many tools, so users will almost always want blanket defer-loading with at most a few exceptions.

Your proposed design is better for that common case. Today, with 50 tools you'd wrap all 50 in SearchableAIFunctionDeclaration; with the proposed design, you'd just add new HostedToolSearchTool() and you're done. Much cleaner.

A few thoughts on the details:

Enable/Disable semantics: The logic makes sense and is consistent with how HostedMcpServerTool.AllowedTools works (null = all allowed). I wonder if the naming could be more specific though — maybe DeferredTools / NonDeferredTools, or SearchableTools / AlwaysLoadedTools? The Enable/Disable naming is generic and doesn't hint at what is being enabled. But that's a bikeshed.

String names vs AITool references: String names (like AllowedTools uses) are simpler and avoid lifetime/reference issues, but AITool references give type safety. Either could work; strings are probably simpler since the provider just needs to match tool names when iterating ChatOptions.Tools.

Namespace: The current design has per-tool Namespace. If we consolidate into HostedToolSearchTool, we'd need either: (a) a single Namespace property on HostedToolSearchTool that applies to all searchable tools, (b) a Dictionary<string, IList<string>> mapping namespaces to tool names, or (c) drop namespace for now and add it later if needed. I'd lean toward (a) or (c) — a single namespace is probably fine for most uses, and (c) keeps the API surface minimal while we learn more.

What we'd lose: The GetService<SearchableAIFunctionDeclaration>() discovery pattern, but that's only useful in middleware, and middleware doesn't typically need to know about defer-loading (it's a wire-level optimization). DelegatingAIFunctionDeclaration would also no longer need to be public (can revert to internal).

Overall I think this is a better design. Want me to implement it?

[stephentoub]

Want me to implement it?

@copilot, yes, please

[Copilot]

Want me to implement it?

@copilot, yes, please

Implemented in f295d10.

Changes:

• HostedToolSearchTool now has DeferredTools and NonDeferredTools (IList<string>?) properties with the enable/disable semantics you described (both null = all deferred)
• Removed SearchableAIFunctionDeclaration entirely
• Reverted DelegatingAIFunctionDeclaration to internal
• OpenAI provider finds HostedToolSearchTool from ChatOptions.Tools and applies defer_loading: true to matching function tools
• Used AOT-safe ModelReaderWriter.Read<ResponseTool>(..., OpenAIContext.Default) and "true"u8 literal (no pragma suppressions)
• Added ChatOptions? parameter to AsOpenAIResponseTool extension (follows AsOpenAIResponseTextFormat pattern)

[stephentoub]

@PederHP any thoughts?

[Copilot]

Pull request overview

Adds a new experimental hosted tool marker (HostedToolSearchTool) to enable “tool_search” and implements deferred loading (defer_loading) support in the OpenAI Responses provider by patching function tools based on DeferredTools / NonDeferredTools configuration.

Changes:

• Introduces HostedToolSearchTool (experimental) and a new DiagnosticIds.Experiments.AIToolSearch constant.
• Updates OpenAI Responses tool conversion to emit tool_search and optionally patch function tools with defer_loading: true.
• Adds/extends unit tests covering tool_search conversion, caching, and deferred-loading selection logic.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
test/Libraries/Microsoft.Extensions.AI.OpenAI.Tests/OpenAIConversionTests.cs	Adds OpenAI conversion tests for tool_search and defer_loading behavior.
test/Libraries/Microsoft.Extensions.AI.Abstractions.Tests/Tools/HostedToolSearchToolTests.cs	Adds tests for HostedToolSearchTool construction and list properties.
src/Shared/DiagnosticIds/DiagnosticIds.cs	Adds DiagnosticIds.Experiments.AIToolSearch.
src/Libraries/Microsoft.Extensions.AI.OpenAI/OpenAIResponsesChatClient.cs	Adds tool_search mapping and deferred-loading patching for function tools.
src/Libraries/Microsoft.Extensions.AI.OpenAI/MicrosoftExtensionsAIResponsesExtensions.cs	Updates AsOpenAIResponseTool to accept ChatOptions? for deferred-loading context.
src/Libraries/Microsoft.Extensions.AI.OpenAI/Microsoft.Extensions.AI.OpenAI.json	Updates API baseline for the modified extension method signature.
src/Libraries/Microsoft.Extensions.AI.Abstractions/Tools/HostedToolSearchTool.cs	Adds the new experimental HostedToolSearchTool abstraction with configuration lists.

[Copilot]

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 5 comments.

[Copilot]

HostedToolSearchTool is intended to be experimental per the PR description, but the type currently has no [Experimental(...)] attribute. This makes the API appear Stable (see the generated Abstractions.json) and could unintentionally lock the API surface. Add [Experimental(DiagnosticIds.Experiments.AIToolSearch, UrlFormat = DiagnosticIds.UrlFormat)] and ensure the corresponding DiagnosticIds constant exists.

[Copilot]

SearchableAIFunctionDeclaration is described as an experimental API, but it is currently missing an [Experimental(...)] attribute. Without it, the API is treated as Stable (and the generated Abstractions.json reflects that), which is hard to undo later. Mark it [Experimental(DiagnosticIds.Experiments.AIToolSearch, UrlFormat = DiagnosticIds.UrlFormat)] and ensure DiagnosticIds.Experiments.AIToolSearch is defined.

[jozkee]

@copilot revise the PR description. This shouldn't be experimental.

[Copilot]

Updated the PR description in 89d8df4 — removed all references to [Experimental] annotations. The types are stable.

[Copilot]

The PR description was updated previously to state all types are stable (not experimental). The current description reflects this — no [Experimental] attributes on the new types.

[jozkee]

FWIW, we can't do anything with SearchableAIFunctionDeclaration.Namespace here, meaning is useless for AsOpenAIResponseTool.

[Copilot]

Removed Namespace entirely from SearchableAIFunctionDeclaration in 89d8df4 — it was indeed unusable in the public AsOpenAIResponseTool API and the grouping approach was wrong for OpenAI's array-based namespace model.

[jozkee]

@copilot revert 89d8df4, we want to keep namespaces.

[Copilot]

Reverted in 0579b05 — Namespace property, ToNamespaceResponseTool, namespace grouping logic, and all associated tests are restored.

[Copilot]

@copilot address feedback

All feedback addressed in 89d8df4:

• Removed Namespace from SearchableAIFunctionDeclaration and namespace grouping from OpenAI provider
• Updated PR description to clarify types are not experimental
• Updated API baselines and tests

[rogerbarreto]

I'm not seeing the immediate benefit of having the Declaration outside of the Tool, seems a bit less obvious from the consumption perspective,

I would expect that by definition when you use a HostedToolSearchTool all functions in it will be deferred resulting in a simplified consumption like below.

What am I missing?

// Wrap specific functions as searchable (deferred loading)
var options = new ChatOptions
{
    Tools =
    [
        new HostedToolSearchTool([getWeather, getForecast]),
        importantTool, // not wrapped — sent without defer_loading
    ]
};

// No CreateToolSet needed as they are all wrapped
var tools = [new HostedToolSearchTool([getWeather, getForecast]);

var response = await chatClient.GetResponseAsync(messages, new() { Tools = tools });

[jozkee]

That was at one point the approach taken in this PR #7377 (comment).

Having new HostedToolSearchTool([getWeather, getForecast]) is less ergonomic for namespaces, only way to define them would be to include another param and if you want to define multiple namespaces, you would then need to have multiple HostedToolSearchTool instances (which we could handle by emitting only one tool_search on the wire).

[jozkee]

Also, having a separate way to mark a function with defer_loading is useful if we ever want to add support for local tool search.

https://developers.openai.com/api/docs/guides/tools-tool-search#client-executed-tool-search
https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool#custom-tool-search-implementation

[jozkee]

@copilot fix CI build errors

[Copilot]

@copilot fix CI build errors

Fixed in 3a28bd0 — the build error was caused by var (ns, functions) in namespaces deconstruction of KeyValuePair, which is not available on net462/netstandard2.0. Replaced with explicit kvp.Key/kvp.Value access. Also restored the @namespace parameter naming that was lost during the revert.

[rogerbarreto]

Having new HostedToolSearchTool([getWeather, getForecast]) is less ergonomic for namespaces, only way to define them would be to include another param and if you want to define multiple namespaces, you would then need to have multiple HostedToolSearchTool instances (which we could handle by emitting only one tool_search on the wire).

And that would also IMO make more sense, so you provide per SearchTool what is the scope (namespace + tools), if you have multiple scopes you provide multiple HostedSearchTools for each, which make sense for me, this would work well, if you also have a customized search tool running in the client side, limited to a specific set of tools.

This approach would avoid incurring in scenarios where you advertise tools but forgot to provide the sibling HostedSearchTool that matches your decorated tools for a specific namespace.

[jozkee]

How would new HostedToolSearchTool([getWeather, getForecast]) interact with a break-glass tool_search? I think deduplication wouldn't be possible (or too hard) and we will end up sending multiple tool_search tools.

How would you enable HostedToolSearchTool with MCP servers?

[jozkee]

@rogerbarreto ptal at #7471