refactor: migrate ai-groq + ai-openrouter onto @tanstack/ai-openai-compatible (#543)#545
Open
tombeckenham wants to merge 35 commits into
Conversation
…543) Adds protected `callChatCompletion`, `callChatCompletionStream`, `extractReasoning`, and `transformStructuredOutput` hooks to `OpenAICompatibleChatCompletionsTextAdapter` so providers with non-OpenAI SDK shapes can reuse the shared stream accumulator, partial-JSON tool-call buffer, RUN_ERROR taxonomy, and lifecycle gates. ai-groq drops `groq-sdk` in favour of the OpenAI SDK pointed at api.groq.com/openai/v1; ai-openrouter keeps `@openrouter/sdk` via hook overrides. ai-ollama remains on BaseTextAdapter (native API has a different wire format). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
🚀 Changeset Version Preview6 package(s) bumped directly, 22 bumped as dependents. 🟨 Minor bumps
🟩 Patch bumps
|
|
| Command | Status | Duration | Result |
|---|---|---|---|
nx affected --targets=test:sherif,test:knip,tes... |
❌ Failed | 6m 17s | View ↗ |
nx run-many --targets=build --exclude=examples/** |
✅ Succeeded | 1m 50s | View ↗ |
☁️ Nx Cloud last updated this comment at 2026-05-11 12:34:06 UTC
@tanstack/ai
@tanstack/ai-anthropic
@tanstack/ai-client
@tanstack/ai-code-mode
@tanstack/ai-code-mode-skills
@tanstack/ai-devtools-core
@tanstack/ai-elevenlabs
@tanstack/ai-event-client
@tanstack/ai-fal
@tanstack/ai-gemini
@tanstack/ai-grok
@tanstack/ai-groq
@tanstack/ai-isolate-cloudflare
@tanstack/ai-isolate-node
@tanstack/ai-isolate-quickjs
@tanstack/ai-ollama
@tanstack/ai-openai
@tanstack/ai-openai-compatible
@tanstack/ai-openrouter
@tanstack/ai-preact
@tanstack/ai-react
@tanstack/ai-react-ui
@tanstack/ai-solid
@tanstack/ai-solid-ui
@tanstack/ai-svelte
@tanstack/ai-utils
@tanstack/ai-vue
@tanstack/ai-vue-ui
@tanstack/preact-ai-devtools
@tanstack/react-ai-devtools
@tanstack/solid-ai-devtools
commit: |
…ons migration Addresses regressions and pre-existing silent failures surfaced by reviewing #545: - `@tanstack/ai`: `toRunErrorPayload` normalizes `AbortError` / `APIUserAbortError` / `RequestAbortedError` to `{ code: 'aborted' }` so consumers can discriminate user-initiated cancellation without matching provider-specific message strings. - `@tanstack/openai-base`: `structuredOutput` throws a distinct "response contained no content" error instead of cascading into a misleading JSON-parse error on an empty string; the post-loop tool-args drain now logs malformed JSON via `logger.errors` so truncated streams don't silently invoke tools with `{}`. - `@tanstack/ai-openrouter`: `stream_options.include_usage` is camelCased to `includeUsage` (Zod was silently stripping it, leaving `RUN_FINISHED.usage` always undefined on streaming); mid-stream `chunk.error.code` is stringified so provider codes (401/429/500) survive `toRunErrorPayload`; assistant `toolCalls[].function.arguments` is stringified to match the SDK's `string` contract; `convertMessage` now mirrors the base's fail-loud guards (empty user content, unsupported content parts). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds OpenRouterResponsesTextAdapter on top of @tanstack/openai-base's responses-text base, mirroring the chat-completions migration in #543. - openai-base: protected `callResponse` / `callResponseStream` hooks on OpenAICompatibleResponsesTextAdapter parallel to the existing `callChatCompletion*` hooks, so providers whose SDK has a different call shape can override without forking processStreamChunks. Re-exports the OpenAI Responses SDK types subclasses need. - ai-openrouter: new OpenRouterResponsesTextAdapter routing through `client.beta.responses.send({ responsesRequest })`. Emits the SDK's camelCase TS shape directly via overrides of convertMessagesToInput / convertContentPartToInput / mapOptionsToRequest, annotated with `Pick<ResponsesRequest, ...>` so future SDK field renames break the build instead of silently producing Zod-stripped wire payloads. Bridges inbound stream events camel -> snake so the base's processStreamChunks reads documented fields unchanged. - Function tools only in v1; webSearchTool() throws with a clear error pointing at the chat-completions adapter. - Folds in the silent-failure lessons from 0171b18 (stringified error codes, stringified tool-call arguments, fail-loud on empty user content). - E2E: new `openrouter-responses` provider slot in feature-support / test-matrix / providers / types / api.summarize, reusing aimock's native `/v1/responses` handler. - 10 new unit tests covering request mapping (snake -> camel for top-level fields, function-call camelCasing in input[], variant suffix), stream-event bridge (text deltas, function-call lifecycle, response.failed, top-level error code stringification), webSearchTool() rejection, and SDK constructor wiring. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Removes `validateTextProviderOptions` (no-op stub never called) and the chain of `ChatCompletion*MessageParam` / `ChatCompletionContentPart*` / `ChatCompletionMessageToolCall` types that were only referenced by it. Unblocks the root `test:knip` CI check. None of the removed exports are re-exported from the package's public `src/index.ts`, so this is internal-only cleanup. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The OpenRouter SDK's stream-event schema is built with Speakeasy's
discriminated-union helper, which on a per-variant parse failure falls
back to `{ raw, type: 'UNKNOWN', isUnknown: true }` rather than throwing.
This happens whenever an upstream omits an "optional-looking" required
field — notably `sequence_number` and `logprobs` on text/reasoning delta
events, which aimock-served fixtures don't include.
Before this fix the adapter's switch hit the default branch for UNKNOWN
events and emitted them with no usable `type`, so the base's
processStreamChunks ignored them silently — the run terminated as
`RUN_FINISHED { finishReason: 'stop' }` with no content.
The `raw` payload preserved on the fallback is the original wire-shape
event in snake_case, which is exactly what processStreamChunks reads.
Re-emit it verbatim. Real-OpenRouter responses still flow through the
existing camel -> snake bridge because their events include the required
fields and parse cleanly.
Unblocks the openrouter-responses E2E suite: 11 affected tests now pass
locally against aimock; before this commit they all timed out empty.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…i-openrouter-ai-ollama-to-openai-base-+-parameterize-the-base-for-sdk-shape-variance
Replaces ~200 sites of `asChunk({ type: 'X', ... })` (a `Record<string,
unknown> as unknown as StreamChunk` cast) with `({ type: EventType.X, ... })
satisfies StreamChunk` so the type system validates AG-UI event shape at
every emission. The cast was bypassing TypeScript's string-enum nominal
typing and masking a cluster of spec deviations now fixed:
- RUN_STARTED / RUN_FINISHED in openai-base (chat-completions + responses)
and all three summarize adapters were missing the AG-UI-required
`threadId`. Threading `options.threadId ?? generateId(this.name)` through
`aguiState` (matching the existing Gemini/Anthropic pattern) fixes it.
- RUN_ERROR emissions carried a non-existent `runId` field and the
deprecated nested `error: { message, code }` form instead of AG-UI's
top-level `message`/`code`. Both forms now coexist (deprecated kept for
back-compat) and `runId` is dropped — verified no consumer reads it
(chat-client.ts:404 only reads runId on RUN_FINISHED).
- STEP_STARTED / STEP_FINISHED in responses-text.ts were passing only the
deprecated `stepId` alias; AG-UI requires `stepName`. Now passes both.
- `finishReason` in chat-completions-text.ts was typed as `string`,
dropping below the AG-UI vocabulary. Widened `RunFinishedEvent.finishReason`
in `@tanstack/ai` to include OpenAI's `'function_call'` so it narrows
cleanly. responses-text.ts maps Responses-API `'max_output_tokens'` →
`'length'` and passes `'content_filter'` through.
- Per-event timestamps. AG-UI spec: "Optional timestamp indicating when
the event was created." Previously a single `const timestamp = Date.now()`
was captured at run start and reused on every emission across the eight
adapters; each chunk now uses `Date.now()` inline.
`@tanstack/ai/tests/test-utils.ts` `ev.*` builders are typed to return
precise event members via `satisfies StreamChunk`; the loose `chunk(type,
fields)` factory is preserved as a documented escape hatch for tests that
deliberately construct off-spec fixtures. ai-client tests no longer declare
a local `asChunk`. ai-groq's `processStreamChunks` override signature is
updated to include the new `threadId` field on `aguiState`.
Out of scope, flagged for follow-up:
- Framework tests (ai-react / ai-svelte / ai-vue) with inline string-literal
chunk arrays — their test directories aren't currently type-checked, so
they compile despite being off-spec.
- Summarize adapters omit TEXT_MESSAGE_START / TEXT_MESSAGE_END around
content emissions (separate AG-UI lifecycle gap).
Verified: pnpm -r test:types, test:lib, test:eslint, test:build all green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
tombeckenham
added a commit
that referenced
this pull request
May 12, 2026
#545's asChunk removal added \`threadId\` to RUN_STARTED/RUN_FINISHED on the chatStream path. The structuredOutputStream lift on this branch was emitting those events without \`threadId\`; the new \`satisfies StreamChunk\` checks now catch it. Plumb \`threadId\` through structuredOutputStream's aguiState in both bases. Also drop the residual \`asChunk()\` wrappers in my structuredOutputStream yields and use \`type: EventType.X, ... } satisfies StreamChunk\` directly, matching #545's new convention. While we're here: the chat-completions \`processStreamChunks\` finalisation forwards the SDK's \`finish_reason\` directly into \`RUN_FINISHED.finishReason\`, but the SDK type still includes the legacy \`function_call\` value that AG-UI doesn't accept. #545's \`satisfies\` cleanup exposed the mismatch — collapse \`function_call\` to \`stop\` alongside the existing orphan \`tool_calls\` collapse.
The chat adapter's convertMessage JSON-stringified Array<ContentPart> assistant content (so a multi-part assistant turn would round-trip as the literal JSON of the parts instead of joined text) and emitted `content: undefined` for tool-call-only assistants where the OpenAI Chat Completions contract documents `null`. Use the base's extractTextContent + emit `null` for the tool-call-only case so the override matches the chat-completions base. The Responses adapter's convertMessagesToInput tool branch had the same shape — JSON.stringify(message.content) fed the raw ContentPart shape into function_call_output.output for structured tool results. Use extractTextContent there too. Regression tests assert (a) array-shaped assistant content extracts to joined text rather than JSON, and (b) tool-call-only assistant content emits `null` rather than `undefined`.
The interface declared a single capitalized `Function` key with no
`type` discriminator. The OpenAI / Groq Chat Completions wire format
for a named tool_choice is `{ type: 'function', function: { name } }`.
Construct a literal against the old type and the SDK's Zod schema
would either reject it or treat tool_choice as unset.
No production code constructs this type literally yet — only the
`ChatCompletionToolChoiceOption` union in the same file uses it — so
fixing the shape now is a no-op at runtime but locks the type to the
correct contract going forward.
The module-level pendingMockCreate is only cleared inside applyPendingMock when a factory call consumes it. Tests in the first describe block instantiate the adapter without calling setupMockSdkClient first, so a leaked value from a prior test would inject a stale mock into a later adapter. Reset in beforeEach for deterministic ordering regardless of test-runner permutation.
The feature-support matrix advertises summarize / summarize-stream for both `openrouter` and `openrouter-responses`, but the factories silently substituted `createOpenaiSummarize` against the OpenAI base URL — exercising the OpenAI adapter while reporting OpenRouter coverage. Wire `createOpenRouterSummarize` (a thin wrapper over the OpenRouter chat adapter, used for both rows since the summarize endpoint is chat-completions-only) against the LLMOCK base so the matrix's claim is actually verified.
Sibling adapters (`ai-openai`, `ai-groq`, `ai-grok`) all declare zod as a peerDependency so a consumer that passes a Zod tool schema gets a single zod instance shared with this adapter. Without the peerDep, strict installs (pnpm `strict-peer-dependencies`, yarn berry pnp) can end up with two zod copies — one transitive via `@openrouter/sdk` or `@tanstack/ai`, one direct — and `instanceof ZodType` checks then fail across the boundary.
…override The Groq subclass declared its aguiState parameter with an extra `timestamp: number` field that does not exist on the base class's aguiState type. TypeScript's bivariant method-parameter checks let the wider type pass typecheck, but at runtime the body never reads `timestamp` and the field is never populated by the base, so any caller (or future override) that relied on the declared shape would observe `undefined`. Realign the override's parameter type with the base.
The chunk-level 'error' branch in adaptOpenRouterResponsesStreamEvents already stringifies provider codes so they survive toRunErrorPayload's string-only code filter, but the parallel response.failed / response.incomplete path went through toSnakeResponseResult which forwarded `r.error.code` raw. A provider that returned a numeric code (401/429/500/…) on a terminal failure event would lose it on the way through to RUN_ERROR. Mirror the chunk-level stringification inside toSnakeResponseResult and add a regression test for response.failed with a numeric error.code.
When a base64 image source has no mimeType the override produced a literal `data:undefined;base64,...` URI that the upstream rejects as invalid. The chat-completions base defaults to `application/octet-stream` for exactly this case; mirror the same defaulting in the OpenRouter convertContentPart override. Regression test asserts the data URI no longer contains the literal `undefined`.
The Responses adapter's processStreamChunks marked `runFinishedEmitted` on a top-level chunk.type === 'error' to prevent the synthetic terminal block from firing, but it did not return from the for-await loop. Any subsequent chunks the upstream delivered after a terminal error event (a stray output_text.delta, an output_item.done, etc.) would continue to emit lifecycle events past RUN_ERROR, violating the 'RUN_ERROR is terminal' contract. Mirror the response.failed / response.incomplete branches above: return after yielding RUN_ERROR. Regression test covers the case where the upstream continues delivering chunks after a top-level error event and asserts no further chunks reach the consumer.
…ough transformStructuredOutput hook The Responses base hard-coded transformNullsToUndefined on parsed structured-output JSON, leaving no hook for subclasses to opt out. The changeset's promise of 'transformStructuredOutput for subclasses (like OpenRouter) that preserve nulls in structured output instead of converting them to undefined' was therefore only fulfilled on the chat-completions surface — the matching Responses adapter would silently strip nulls regardless of provider intent. Add the transformStructuredOutput protected hook on OpenAICompatibleResponsesTextAdapter mirroring the chat-completions base's design, and override it as a no-op on OpenRouterResponsesTextAdapter so OpenRouter callers see null sentinels round-trip identically across the two adapter surfaces. Regression test asserts a structuredOutput response containing `nickname: null` round-trips as null (not undefined) through the Responses adapter.
The chat-completions adapter's convertMessage tool branch still
JSON-stringified Array<ContentPart> tool message content, so a tool
result delivered as structured parts (e.g. [{type:'text', content:
'"temp":'}, {type:'text', content:'72'}]) reached the model as the
literal JSON of the parts rather than the joined textual result. The
parallel responses adapter override was fixed earlier; this mirrors
the same fix on the chat-completions path so both surfaces handle
structured tool content identically.
Regression test feeds a structured tool result and asserts the wire
payload's tool message content is the joined text without any
'"type":"text"' leakage.
Every sibling adapter (ai-openai, ai-grok, ai-openrouter, ai-anthropic, ai-gemini, ai-fal, ai-ollama) explicitly lists `@tanstack/ai: workspace:*` under devDependencies in addition to declaring it as a peer. ai-groq omitted the devDep entry, so resolution worked only via pnpm's autoInstallPeers behaviour — toggling that off (strict installs, some yarn berry configs) would silently break ai-groq while every other adapter kept working. Add the dev dep for parity.
…ions
The chat-completions OpenRouter adapter's convertContentPart for audio
unconditionally emitted `{ type: 'input_audio', inputAudio: { data,
format: 'mp3' } }` — but `data` is supposed to be base64. A
URL-sourced audio part therefore shipped the literal URL string into
the base64 slot, which the upstream rejects (or worse, treats as
garbage audio bytes). The Responses adapter already handles this by
routing URL audio through `input_file` (where the URL belongs);
chat-completions has no `input_file` shape on this surface, so
mirror the existing document fallback: emit a text reference noting
the URL. Callers needing real audio URL support should use the
Responses adapter.
The header comment claimed these types "mirror the Groq SDK types", but the migration dropped the groq-sdk dependency entirely in favour of pointing the OpenAI SDK at Groq's /openai/v1 base URL. The file is now the source of truth for Groq-specific wire fields (compound tools, citation/service-tier provider options, …), not a mirror of an external SDK. Update the header to reflect the post-migration role.
The chat-completions convertContentPart 'document' branch unconditionally
returned `{ type: 'text', text: `[Document: ${part.source.value}]` }`.
For URL sources that's a reasonable degradation. For data sources,
`part.source.value` is the raw base64 payload — a multi-megabyte
document would be inlined into the prompt verbatim, blowing the
context window and leaking the document content as plaintext bytes.
Branch on `part.source.type`: URL sources keep the text-reference
fallback, data sources throw with a clear error pointing the caller at
the Responses adapter (which has proper `input_file` support for
inline document data). Mirrors the audio URL/data branching added in
the prior round.
The "base" name implied this package tracked OpenAI's product roadmap. In reality it implements two OpenAI-shaped wire-format protocols (Chat Completions, Responses) that multiple providers ship — OpenRouter, Groq, Grok, vLLM, SGLang, Together. "OpenAI-compatible" is the industry term for this family (cf. Vercel's @ai-sdk/openai-compatible, LiteLLM, BentoML, Lightning AI). OpenRouter's beta Responses endpoint routes to Claude, Gemini, and other underlying models, confirming that /v1/responses (like /v1/chat/completions) is a multi-provider wire format rather than OpenAI-only — so the Responses adapter stays alongside Chat Completions in the renamed package. Pure rename, no behavior change. Class names (OpenAICompatibleChatCompletionsTextAdapter, OpenAICompatibleResponsesTextAdapter, …) and protected hook contracts are unchanged. Consumer packages (ai-openai, ai-openrouter, ai-groq, ai-grok) only update internal import paths; public API is unchanged. @tanstack/openai-base@0.2.x remains published on npm for any pinned lockfile references but will receive no further updates. A README in the renamed package documents the protocol-vs-product contract. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…mpatible Match the `ai-*` prefix convention used by every other package in the AI subnamespace (ai-utils, ai-openai, ai-anthropic, ai-client, ai-react, …). `@tanstack/` is a flat namespace shared across all TanStack products (Query, Router, Table, Form, AI, …), so `@tanstack/openai-compatible` alone gives no signal about which TanStack product it belongs to. Pure rename of the rename in 06d3d8c; no behavior change. Directory `packages/typescript/openai-compatible` → `packages/typescript/ai-openai-compatible`, package.json `name` field, consumer dependency declarations, TypeScript imports, README, CHANGELOG header, both changesets all updated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…oduct framing
Rewrite the `@tanstack/ai-openai-compatible` README to lead with the thinking — OpenAI authored two wire formats (Chat Completions and Responses) that many vendors implement, so the package contains the shared logic for talking to any server that speaks one of those wire formats, not "the base for OpenAI." Add a side-by-side table for "what goes here vs in @tanstack/ai-openai" and a contributor rule of thumb ("a field belongs here only if at least two compatible providers support it").
Expand the leading docstrings on both OpenRouter text adapters to explicitly answer "why does this extend from @tanstack/ai-openai-compatible?" — OpenRouter implements OpenAI's wire formats verbatim (Chat Completions natively, Responses as a beta routing layer that fans out to Claude/Gemini/etc.), so the shared base lets us inherit ~1k LOC of stream accumulation, partial-JSON buffering, AG-UI lifecycle, and structured-output coercion rather than duplicating it.
No code change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
tombeckenham
changed the title
…ection The OpenRouter adapter package now ships two adapters (`openRouterText` for /v1/chat/completions and `openRouterResponsesText` for /v1/responses beta). Document the difference for consumers: both route to any underlying model in the catalogue (Anthropic, Google, Meta, …); the wire format describes the client → OpenRouter call, not which provider answers. Add a side-by-side table, a basic example for the Responses adapter, and beta caveats (no branded server-tools yet; prefer the chat-completions adapter if in doubt). No mention of the internal shared-base package — that's an implementation detail consumers don't need to track. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…om openai SDK Make `OpenAICompatibleChatCompletionsTextAdapter` and `OpenAICompatibleResponsesTextAdapter` abstract. Subclasses now own SDK client construction and implement the `callChatCompletion*` / `callResponse*` hooks. The base never imports `openai` at runtime — only as types — so `dist/esm/*.js` is openai-free and the package's `openai` dep moves to optional `peerDependencies` + `devDependencies`. Delete image/tts/transcription/video bases (single-user; only ai-openai extended them, so they're now standalone classes there). Move summarize to `@tanstack/ai` core as `ChatStreamSummarizeAdapter` — it's protocol-agnostic, wraps any `ChatStreamCapable`. Provider-specific `*SummarizeAdapter` classes deleted, replaced by thin factory functions returning `ChatStreamSummarizeAdapter` directly. ai-grok duplicates its image adapter standalone (~150 LOC; shared base wasn't worth the indirection for thin SDK wrappers). Also fix the round-3 double `TOOL_CALL_END` regression in the Responses adapter: `function_call_arguments.done` now gates on `!metadata.ended` so the output_item.done backfill path can't emit a duplicate close. Regression test added. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… wrapper - Migrate anthropic, gemini, ollama, openrouter summarize adapters to thin factories over ChatStreamSummarizeAdapter, matching the openai/grok pattern. Drops ~600 lines of duplicated streaming/error/usage handling. - Thread modelOptions from SummarizationOptions through the activity layer and into the wrapped text adapter's chatStream so provider-specific knobs (cache control, plugins, safety settings, tuning params) reach the wire. - Add InferTextProviderOptions<TAdapter> helper to extract per-model provider options from a text adapter's ~types. - Drop bespoke XSummarizeProviderOptions interfaces from all 6 providers; provider summarize types now resolve to the text adapter's per-model options shape, giving accurate IntelliSense for modelOptions. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🎯 Changes
Closes #543 (for groq + openrouter; ollama stays on
BaseTextAdapter— see "Out of scope" below).This PR grew three additional layers on top of the original migration. Sections in execution order.
1. Migration (original scope)
@tanstack/ai-openai-compatible(the shared base — see §3 for the name) adds four protected hooks onOpenAICompatibleChatCompletionsTextAdapterso providers with non-OpenAI SDK shapes can plug in:callChatCompletion/callChatCompletionStream(SDK call sites),extractReasoning(surface reasoning content into the sharedREASONING_*+ legacySTEP_STARTED/STEP_FINISHEDlifecycle), andtransformStructuredOutput(subclasses can opt out of the default null→undefined transform). Defaults preserve existing behaviour forai-openai/ai-grok.@tanstack/ai-groqrewritten as a thin subclass (~91 LOC, down from 650). Dropsgroq-sdkin favour of the OpenAI SDK pointed athttps://api.groq.com/openai/v1. Preserves thex_groq.usagequirk via a smallprocessStreamChunkswrapper.@tanstack/ai-openrouterrewritten as a subclass with hook overrides (~396 LOC, down from 807). Keeps@openrouter/sdkfor typed provider routing, plugins, and metadata; a small request shape converter (max_tokens→maxCompletionTokens, etc.) and chunk shape adapter bridge the SDK boundary.OpenRouterResponsesTextAdapter) that extendsOpenAICompatibleResponsesTextAdapter.Net: −731 lines at original-migration time, ~1k LOC of duplicated stream/tool/lifecycle code removed.
2. Edge-case fixes surfaced during review
RequestAbortedError/APIUserAbortError/AbortErrorall normalize to a stable{ code: 'aborted' }payload intoRunErrorPayload, so consumers can discriminate user-initiated cancellation without matching provider-specific message strings.stream_options.include_usagecorrectly camelCased toincludeUsageso streamingRUN_FINISHED.usageis populated.structuredOutputthrows a distinct"response contained no content"error rather than letting empty content cascade into a misleading JSON-parse error.logger.errors, so truncated streams emitting partial tool args are debuggable instead of silently invoking the tool with{}.toolCalls[].function.argumentsstringified to match SDK contract;convertMessagemirrors the base's fail-loud guards (throws on empty user content and unsupported content parts); audio URLs route to text fallback; inline document data rejected; assistant/tool message content fidelity preserved; image data URI mime defaults tooctet-stream;error.codestringified onresponse.failed.3. Package rename:
@tanstack/openai-base→@tanstack/ai-openai-compatibleThe migration originally shipped
@tanstack/openai-base. While reviewing we caught that the name was wrong on two axes:/v1/chat/completionsas their native API (Groq, Together, vLLM, SGLang, Fireworks, Ollama's compat layer); OpenRouter's beta/v1/responsesroutes to Anthropic Claude and Google Gemini under an OpenAI-shaped wire format. Both endpoints are multi-vendor protocols, not OpenAI-only product surfaces.@tanstack/is a flat namespace shared across all TanStack subproducts.@tanstack/openai-basegave no signal about which subproduct it belonged to.Renamed to
@tanstack/ai-openai-compatible: "OpenAI-compatible" matches industry terminology (Vercel's@ai-sdk/openai-compatible, LiteLLM, BentoML); theai-prefix matches every other AI subnamespace package.@tanstack/openai-base@0.2.xremains published on npm for any pinned lockfile references but will receive no further updates.4. Narrowing
ai-openai-compatibleto its purposeDecoupled
ai-openai-compatiblefrom the OpenAI SDK type surface and narrowed its export footprint to just the two protocol adapters (Chat Completions, Responses). The package no longer re-exports OpenAI SDK types or non-protocol helpers; those stay inai-openai. README documents the contributor rule of thumb: a field belongs inai-openai-compatibleonly if at least two compatible providers support it; otherwise it belongs in the provider's own package.5. Summarize adapter consolidation (all 6 providers)
Migrated the remaining provider summarize adapters (anthropic, gemini, ollama, openrouter) onto
ChatStreamSummarizeAdapter— openai and grok already used this pattern. Each provider'ssummarize.tscollapses from 200–300 lines of bespoke streaming/error/usage handling to a ~50-line factory.Plus:
modelOptionsfromSummarizationOptionsthrough the activity layer and into the wrapped text adapter'schatStream, so provider-specific knobs (Anthropic cache control, OpenRouter plugins, Ollama tuning, Gemini safety settings) actually reach the wire on summarize calls.InferTextProviderOptions<TAdapter>helper in@tanstack/ai/adaptersto extract per-model provider options from a text adapter's~types.XSummarizeProviderOptionsinterfaces from all 6 providers — they previously claimed minimal{ temperature?, maxTokens? }shapes that didn't reflect what the text adapter actually accepted. Provider summarize types now resolve to the text adapter's per-model options shape, giving accurate IntelliSense formodelOptions.AnthropicSummarizeAdapter,GeminiSummarizeAdapter,OllamaSummarizeAdapter,OpenRouterSummarizeAdapter) from the public surface, matching the openai/grok shape.chat-stream-wrapper.ts→chat-stream-summarize.tsfor clarity.Out of scope
ai-ollamatext remains onBaseTextAdapter. Ollama's native API uses a different wire format from Chat Completions — different chunk shape, request shape, non-incremental tool-call streaming, and a differentformatfield for structured output. The compatible base'sprocessStreamChunks(the bulk of the duplication win) assumes OpenAI Chat Completions chunks; bridging Ollama would require overriding every inherited method, leaving the base doing no useful work. Migrating it remains a separate effort.Why
OpenAICompatibleResponsesTextAdapterstays inai-openai-compatibleAn earlier draft of §3 considered moving the Responses adapter into
@tanstack/ai-openaion the assumption that/v1/responseswas OpenAI-only. OpenRouter's beta routing disproves that — they accept OpenAI's Responses wire format and fan out to any model (Claude, Gemini, …). Responses is younger and currently has fewer native implementers than Chat Completions, but it's the same kind of thing. Moving it toai-openaiwould mis-encode the architecture, force a cross-packageai-openrouter → ai-openaidependency, and split the streaming-JSON / partial-tool-arg logic across two locations when it should stay shared.✅ Checklist
pnpm run test:pr.Specific results:
pnpm test:types— passes for all 8 affected packages (@tanstack/ai,ai-openai,ai-anthropic,ai-gemini,ai-ollama,ai-grok,ai-openrouter,ai-openai-compatible)pnpm test:eslint— passes (1 pre-existing warning, unrelated)pnpm test:lib— 1,208 unit tests pass across the affected packagespnpm test:build(publint) — 60/60 ✔pnpm test:knip— cleanpnpm test:sherif— no issuespnpm changeset:publish --dry-runcorrectly identifies@tanstack/ai-openai-compatible@0.2.1as the only new package to publishManual smoke (gating signal per CLAUDE.md):
x_groq.usageoverride)provider: { order: [...] }(validates provider routing through request-shape conversion):thinkingvariant (validatesextractReasoninghook)RequestAbortedError→ RUN_ERROR mapping)modelOptionspassing a provider-specific knob to verify it reaches the wire🚀 Release Impact
Two changesets in this PR:
migrate-groq-openrouter-to-openai-base.md— migration + hooks + edge-case fixes (§1, §2)rename-openai-base-to-ai-openai-compatible.md— package rename (§3)TODO before publish: the summarize consolidation (§5) drops the
XSummarizeProviderOptionsinterfaces andXSummarizeAdapterclass exports fromai-anthropic,ai-gemini,ai-ollama, andai-openrouter. A follow-up changeset covering that breaking-but-narrow surface change is still pending.🤖 Generated with Claude Code