You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
AI Bridge should behave as a reverse proxy: forward the client's request to the upstream provider as-is, and relay the upstream response back unchanged. Today, bridged routes deserialize requests into SDK types, re-serialize them via SDK methods, and reconstruct responses from SDK structs, all of which adds fragility, breaks unknown fields, and couples us to SDK behavior.
This epic tracks the incremental steps to get there.
Goal
Bridged routes should work like passthrough routes: raw bytes in, raw bytes out, with middleware hooks for header injection, usage recording, and observability.
Steps
Clean up header handling: remove ExtraHeaders mechanism (redundant after Forward client request headers to upstream providers #192), move actor header injection into the header middleware, add X-Forwarded-* headers to bridged routes.
Remove the SDK from the HTTP transport: stop using SDK methods to make upstream HTTP requests. Instead, forward the raw request body to upstream via reverse proxy. The SDK is still used to deserialize the request body (read-only) to extract fields needed for routing, metrics, and recording.
Remove the SDK from request deserialization: Replace SDK type deserialization with lightweight JSON inspection (e.g., gjson) to extract only the fields needed for routing, metrics, and recording. After this step, the SDK is fully out of the request path.
Remove the SDK from the response path: replace SDK response parsing with lightweight JSON inspection to extract fields needed for recording (tokens, tool calls, prompts) from the raw response stream.
Unify bridged and passthrough code paths: converge into a single reverse proxy with per-route middleware for recording and header injection.
Note: Steps 2-5 are blocked on MCP tool injection removal. MCP injection is the main reason the SDK is in the request path today: it requires deserializing the request to inject tools, and deserializing the response to intercept tool calls and run the inner agentic loop. Once MCP injection is removed, there's no longer a need to mutate request or response bodies, and we can proxy raw bytes. Step 1 can proceed independently.
Description
AI Bridge should behave as a reverse proxy: forward the client's request to the upstream provider as-is, and relay the upstream response back unchanged. Today, bridged routes deserialize requests into SDK types, re-serialize them via SDK methods, and reconstruct responses from SDK structs, all of which adds fragility, breaks unknown fields, and couples us to SDK behavior.
This epic tracks the incremental steps to get there.
Goal
Bridged routes should work like passthrough routes: raw bytes in, raw bytes out, with middleware hooks for header injection, usage recording, and observability.
Steps
ExtraHeadersmechanism (redundant after Forward client request headers to upstream providers #192), move actor header injection into the header middleware, addX-Forwarded-*headers to bridged routes.Note: Steps 2-5 are blocked on MCP tool injection removal. MCP injection is the main reason the SDK is in the request path today: it requires deserializing the request to inject tools, and deserializing the response to intercept tool calls and run the inner agentic loop. Once MCP injection is removed, there's no longer a need to mutate request or response bodies, and we can proxy raw bytes. Step 1 can proceed independently.
Related to:
ProcessRequestacross all interception types (closed in favor of this refactor)