Add shared SchemaCache for streamable-http server #1993
Conversation
Create a shared mcp.SchemaCache in RunHTTPServer and pass it through to each per-request MCP Server via ServerOptions. This avoids repeated JSON schema reflection and resolution when a new Server is created for every request in stateless mode, matching the pattern used by the remote server.
There was a problem hiding this comment.
Pull request overview
This PR improves performance in streamable-http (stateless) mode by introducing a shared mcp.SchemaCache that is reused across per-request mcp.Server instances, avoiding repeated JSON schema reflection/resolution work on every request.
Changes:
- Create a single shared
mcp.SchemaCacheduring HTTP server startup and pass it into the handler via a new option. - Extend the HTTP handler options/struct to carry the shared cache and apply it to each per-request
mcp.ServerOptions.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| pkg/http/server.go | Instantiates a shared mcp.SchemaCache once and passes it into the HTTP MCP handler. |
| pkg/http/handler.go | Adds handler/option plumbing for SchemaCache and sets it on each per-request mcp.ServerOptions. |
Co-authored-by: SamMorrowDrums <4811358+SamMorrowDrums@users.noreply.github.com>
| - uses: actions/setup-go@v6 | ||
| with: | ||
| go-version: stable | ||
| go-version: '1.25' |
There was a problem hiding this comment.
Linter requires this
Co-authored-by: SamMorrowDrums <4811358+SamMorrowDrums@users.noreply.github.com>
pkg/http/handler.go
Outdated
| inventoryFactoryFunc: inventoryFactory, | ||
| oauthCfg: opts.OAuthConfig, | ||
| scopeFetcher: scopeFetcher, | ||
| schemaCache: opts.SchemaCache, |
There was a problem hiding this comment.
Do you think there would be any scenario where we don't want a schema cache for HTTP?
I feel like we can drop the WithSchemaCache when initializing (still keeping WithSchemaCache), and init an empty schema cache by default?
I guess the default right now is without a schema cache, but is that a good default? Maybe if you explicitly don't want one being able to pass nil, but that is difficult to manage with the current way that Options work
There was a problem hiding this comment.
Yeah that's a good question, I mean I don't think we want to make it optional, because in the end we're in control and we don't want it pluggable, it's our opinionated server after all. I'll spin off copilot to riff on that and see if we like it.
Co-authored-by: SamMorrowDrums <4811358+SamMorrowDrums@users.noreply.github.com>
Summary
Creates a shared
mcp.SchemaCacheinRunHTTPServerand passes it through to each per-request MCP Server viaServerOptions.SchemaCache.Motivation
In streamable-http (stateless) mode, a new
mcp.Serveris created for every request. Without a shared cache, JSON schema reflection and resolution is repeated on every request for every tool. The go-sdk'sSchemaCache(sync.Map-backed, goroutine-safe) caches these schemas across server instances, matching the pattern already used by the remote server.Changes
pkg/http/server.go: Createmcp.NewSchemaCache()once at startup, pass viaWithSchemaCacheoptionpkg/http/handler.go: AddSchemaCachefield toHandler/HandlerOptions,WithSchemaCacheoption func, and setso.SchemaCacheon per-requestServerOptions