-
Notifications
You must be signed in to change notification settings - Fork 4
DOC-1867 AI Gateway #477
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
DOC-1867 AI Gateway #477
Conversation
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the
📝 WalkthroughWalkthroughAdds extensive AI Gateway documentation and navigation: new AI Gateway index, overview, quickstart, architecture, CEL routing cookbook, observability (logs/metrics), migration guide, MCP aggregation/orchestration guide, administrator and builder setup guides, discovery/connect guides, many IDE/tool integration pages (Claude Code, Cline, Continue.dev, Cursor, GitHub Copilot), MCP Server Patterns/nav updates, a BYOC support partial, and renames the AI Agents index to "Agentic AI". All changes are documentation-only. Sequence Diagram(s)sequenceDiagram
participant Client
participant AI_Gateway as "AI Gateway"
participant MCP_Server as "MCP Server"
participant LLM_Provider as "LLM Provider"
participant Observability as "Observability"
Client->>AI_Gateway: Send request (headers, rp-aigw-id, body)
AI_Gateway->>Observability: Log request + metrics
AI_Gateway->>AI_Gateway: Apply policies (CEL, rate/spend limits, routing)
AI_Gateway->>MCP_Server: Discover/Invoke tools (deferred load / orchestrator)
MCP_Server-->>AI_Gateway: Tool result(s)
AI_Gateway->>LLM_Provider: Forward transformed request (provider pool / fallback)
LLM_Provider-->>AI_Gateway: Response (tokens, cost)
AI_Gateway->>Observability: Log response, costs, routing decisions
AI_Gateway-->>Client: Consolidated response (LLM + MCP tool outputs)
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Comment |
✅ Deploy Preview for rp-cloud ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
|
||
| The orchestrator is a built-in MCP server that enables programmatic tool calling. The agent can generate JavaScript to call multiple tools in a single orchestrated step, which reduces the number of round trips. For example, a workflow requiring 47 file reads can be reduced from 49 round trips to just 1. | ||
|
|
||
| *REVIEWERS: When/how exactly do you use the orchestrator? Also what happens after they create a gateway? Please provide an example of how to validate end-to-end routing against the gateway endpoint!* |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it is an MCP server exposing tools, so it is used as another regular MCP server. I can provide examples.
|
|
||
| This can reduce token usage significantly (for example, 80-90% depending on how many servers/tools are configured). | ||
|
|
||
| == Observability |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I haven't gotten to the observability implementation yet, probably next week.
|
|
||
| This is central to governance: You can see and control usage by gateway boundary (for example, by team, environment, customer, or product). | ||
|
|
||
| *REVIEWERS: Where do those metrics appear in the UI, or how does a user validate observability after setup?* |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is not implemented yet.
9aef822 to
b28bea7
Compare
| ---- | ||
| { | ||
| "claude.mcpServers": { | ||
| "redpanda-ai-gateway": { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This configuration should use http as transport. For example, to configure our internal MCP gateway in Claude Code, it would look roughly like this:
claude mcp add --transport http redpanda-aigateway https://gw.ai.panda.dev/mcp --header "Authorization: Bearer YOUR_API_KEY" --header "rp-aigw-id: GATEWAY_ID"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is also a way to disallow people from using other MCP servers. I'm going to have to dedicate some time to documenting and testing all the integrations with popular AI coding agents.
# Conflicts: # modules/ROOT/nav.adoc
Added 7 new documentation files for AI Gateway: - what-is-ai-gateway.adoc: Overview, problem/solution framing, common patterns - quickstart-enhanced.adoc: Step-by-step quickstart with time markers - observability-logs.adoc: Request logs, filtering, and debugging - observability-metrics.adoc: Dashboards, analytics, and cost tracking - migration-guide.adoc: Safe migration from direct provider integration - cel-routing-cookbook.adoc: CEL routing patterns with examples - mcp-aggregation-guide.adoc: MCP aggregation and orchestration All files follow Redpanda documentation standards: - Sentence case headings - Imperative verbs for action headings - AsciiDoc format - Comprehensive placeholders for product-specific details Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add personas, learning objectives, and prerequisites to all AI Gateway documentation pages. Remove DRAFT prefixes from titles and time estimates from quickstart. Fix passive voice in multiple locations. Changes: - Add page-personas attributes to all 7 files - Add learning objectives in ABCD format - Add prerequisites sections where missing - Remove "DRAFT:" from all page titles - Remove time estimates from quickstart-enhanced.adoc - Fix passive voice constructions - Improve page descriptions - Preserve all placeholder comments for future content Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
1297ef0 to
49a573f
Compare
Provide admin and user guides for configuring Claude Code, Cline, Continue.dev, Cursor IDE, and GitHub Copilot to work with AI Gateway, enabling centralized LLM routing and MCP tool aggregation. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
…oud-team-polish-clean-up # Conflicts: # modules/ROOT/nav.adoc # modules/ai-agents/pages/mcp/index.adoc
Major improvements: - Rename ai-gateway.adoc to gateway-quickstart.adoc for clearer naming - Rename ai-gateway-overview.adoc to gateway-architecture.adoc - Consolidate quickstart-enhanced.adoc into gateway-quickstart.adoc - Remove 161 lines of duplicate content between what-is and architecture pages - Add page metadata (:page-topic-type:, :page-personas:, :learning-objective-N:) to all files - Convert learning objectives from bullets to required attribute format - Create BYOC version requirement partial and add to all 23 AI Gateway pages - Restructure navigation with clearer sections (Overview, Quickstart, Architecture, Observability) - Update all cross-references to renamed files This consolidation reduces content duplication, improves metadata consistency, and provides better content organization for users. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
…docs Updated 13 files to comply with documentation standards for learning objectives: - Added learning objective attributes to metadata (3 files) - Converted all learning objective bullets to checklist format (* [ ]) Files updated: - admin/setup-guide.adoc - builders/connect-your-agent.adoc - builders/discover-gateways.adoc - All 10 integration guide files (admin and user variants for Claude Code, Cline, Continue, Cursor, GitHub Copilot) All 23 AI Gateway files now have proper learning objective format per team standards. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Updated framework integration examples with correct implementations:
1. Anthropic SDK section (lines 238-256):
- Replaced incorrect Anthropic base_url override with OpenAI client
- Anthropic's base_url only works with native /v1/messages API, not
OpenAI-compatible gateways
- Preserved use_gateway env var logic and REDPANDA_AI_GATEWAY_* names
- Removed PLACEHOLDER note
2. LlamaIndex section (lines 629-638):
- Added default_headers alongside additional_kwargs for gateway headers
- Removed PLACEHOLDER note about header syntax
3. Vercel AI SDK section (lines 657-671):
- Changed from openai() provider to createOpenAI() for custom endpoints
- Uses createOpenAI() with baseURL, apiKey, and headers configuration
- Removed PLACEHOLDER note about syntax verification
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Replaced absolute percentage claims with conditional language to present
them as typical or potential outcomes rather than guaranteed results:
1. what-is-ai-gateway.adoc:
- Line 99: "can achieve 99.9% uptime" → "can significantly improve
uptime (potentially up to 99.9% in some configurations)"
- Line 107: "reduces token usage by 80-90%" → "often reduces token
usage by 80-90% depending on your configuration and the number of
tools aggregated"
2. mcp-aggregation-guide.adoc:
- Line 16: "80-90% fewer tokens" → "Often 80-90% fewer tokens
(depending on configuration)"
- Line 152: "80-90% reduction" → "Typically 80-90% reduction"
- Line 269: "Still 80-90% lower" → "Often 80-90% lower"
- Line 335: "Expected Results: 80-90% reduction" → "Typically 80-90%
reduction"
3. gateway-architecture.adoc:
- Line 123: "80-90% token reduction" → "often 80-90% token reduction
depending on configuration"
These changes ensure claims are presented as conditional outcomes that
depend on configuration, rather than absolute guarantees.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Converted AI_GATEWAY_PERSONA_RESTRUCTURING_PLAN.md to proper AsciiDoc format: - Renamed to lowercase: ai-gateway-persona-restructuring-plan.adoc - Converted Markdown headers (# ## ###) to AsciiDoc (= == ===) - Converted Markdown tables to AsciiDoc table format with proper column specs - Converted Markdown code blocks to AsciiDoc [source,text] blocks - Converted Markdown bold (**text**) to AsciiDoc bold (*text*) - Added blank lines before and after all tables and code blocks - Fixed all code blocks to use proper language annotations (text for trees) - Removed old markdown file All tables now have proper blank lines and all fenced code blocks have language annotations, fixing MD058/MD040 violations. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Deleted modules/shared/partials/ai-gateway.png as it's no longer needed. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Deleted ai-gateway-persona-restructuring-plan.adoc as it was an internal planning document that isn't referenced anywhere in the published docs. Planning documents don't need to be in the documentation tree. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 9
🤖 Fix all issues with AI agents
In `@modules/ai-agents/pages/ai-gateway/integrations/claude-code-admin.adoc`:
- Around line 313-355: Update the example config to match Anthropic's MCP
schema: change the file path from "~/.claude/config.json" to the correct
user/state file "~/.claude.json" (or mention project-level ".mcp.json"), remove
the undocumented "apiProviders" section entirely, and update the "mcpServers"
entry to use "type": "http" instead of "transport" (i.e., replace the
"transport" key with "type" and keep "url" and "headers" under the
"redpanda-ai-gateway" object).
In `@modules/ai-agents/pages/ai-gateway/integrations/claude-code-user.adoc`:
- Around line 184-210: The apiProviders example incorrectly shows environment
variable interpolation which only works in the mcpServers section; update the
document so apiProviders does not present "${REDPANDA_GATEWAY_URL}" or
"${REDPANDA_GATEWAY_ID}" as valid interpolation options—either replace those
with sample hardcoded values or add a clear note that only mcpServers supports
${VAR} interpolation and apiProviders requires explicit values or a different
dynamic config mechanism; reference the mcpServers and apiProviders sections and
the REDPANDA_GATEWAY_URL / REDPANDA_GATEWAY_ID symbols so readers know which
fields support interpolation.
In `@modules/ai-agents/pages/ai-gateway/integrations/cline-admin.adoc`:
- Around line 463-471: Replace the token‑like literal in the curl example header
to use an environment variable placeholder instead of "YOUR_API_TOKEN"; update
the Authorization header in the shown curl (the line with -H "Authorization:
Bearer YOUR_API_TOKEN") to something like -H "Authorization: Bearer
$REDPANDA_API_TOKEN" (or use shell expansion via -H "Authorization: Bearer
${REDPANDA_API_TOKEN}") and add a short comment or note that the env var must be
set, so secret scanners won't flag the example.
- Around line 340-383: Remove the unofficial VS Code settings keys
("cline.apiProvider", "cline.apiBaseUrl", "cline.customHeaders") from the
settings.json example and instead document that API/provider configuration is
managed via the Cline extension global state (not VS Code settings); replace the
MCP server example that used "settings.json" and "transport" with an example
that references cline_mcp_settings.json and uses the official fields (use "type"
e.g., "streamableHttp" instead of "transport" and include the proper "url" and
"headers" structure per Cline's remote server schema); also mention the VS Code
toggle name "Cline > Mcp: Mode" as the global MCP enablement setting and
instruct users to configure MCP servers via the Cline UI or
cline_mcp_settings.json rather than settings.json.
In `@modules/ai-agents/pages/ai-gateway/integrations/continue-admin.adoc`:
- Around line 422-517: Update the examples to the current Continue.dev YAML
config standard by renaming the file and examples from ~/.continue/config.json
to ~/.continue/config.yaml and converting the JSON examples to equivalent YAML;
replace the MCP block key experimental.modelContextProtocolServers (plural) with
experimental.modelContextProtocolServer (singular) and ensure each
transport.type uses a valid value (stdio, sse, or streamable-http) instead of
"http"; update apiBase and model/provider entries to YAML syntax (references:
"models", "tabAutocompleteModel", "apiBase", "apiKey") and add guidance to
prefer the .continue/mcpServers/ directory-based MCP configuration method rather
than embedding MCP servers in the main config.yaml.
In `@modules/ai-agents/pages/ai-gateway/integrations/continue-user.adoc`:
- Around line 269-324: The config examples use legacy config.json interpolation
like "${REDPANDA_API_KEY}" which Continue.dev does not expand; replace the JSON
example with an equivalent config.yaml example that uses the correct
interpolation syntax `${{ secrets.REDPANDA_API_KEY }}`, `${{
secrets.REDPANDA_GATEWAY_URL }}`, and `${{ secrets.REDPANDA_GATEWAY_ID }}` for
the keys referenced in the snippet (models -> apiKey/apiBase,
requestOptions.headers.rp-aigw-id, experimental.modelContextProtocolServers ->
transport.url and transport.headers.Authorization/rp-aigw-id), or explicitly
mark config.json as deprecated and add a short note that `${VAR}` will be
treated as a literal string and provide the config.yaml alternative (using the
same fields: models, provider/model, apiKey, apiBase, requestOptions,
experimental.modelContextProtocolServers, transport) so users know to use `${{
secrets.* }}` in config.yaml.
In `@modules/ai-agents/pages/ai-gateway/integrations/cursor-user.adoc`:
- Around line 193-221: The settings.json snippet uses unsupported ${VAR}
interpolation (for keys cursor.overrideOpenAIBaseUrl,
cursor.overrideOpenAIApiKey, and openai.additionalHeaders) so those placeholders
will remain literal; update the doc to either replace the ${VAR} placeholders
with concrete values that users must paste into settings.json, or use the
supported environment-syntax if available (e.g., ${env:VAR}) for those specific
settings, or recommend generating the settings.json at runtime via a small setup
script that reads REDPANDA_GATEWAY_URL, REDPANDA_GATEWAY_ID, and
REDPANDA_API_KEY and writes the actual values into cursor.overrideOpenAIBaseUrl,
cursor.overrideOpenAIApiKey, and openai.additionalHeaders before launching
Cursor.
In `@modules/ai-agents/pages/ai-gateway/integrations/github-copilot-user.adoc`:
- Around line 86-217: The documented settings are incorrect and must be
replaced: change references of github.copilot.advanced.customModels to
github.copilot.chat.customOAIModels (experimental chat OAI models) and remove
the non-existent useOAIProvider usage; replace oai.provider.* examples with the
OAI Compatible Provider extension's actual settings and flow—use
oaicopilot.baseUrl and oaicopilot.models where appropriate and note that API
keys/headers are set via the Copilot Chat → Manage Models UI (not in
settings.json); update the examples and prose to instruct users to open Copilot
Chat → Manage Models to add the OAI Compatible provider, enter base URL, api key
and custom headers there, and then select the configured model in the Copilot
UI.
In `@modules/ai-agents/pages/ai-gateway/migration-guide.adoc`:
- Around line 623-638: The example config duplicates the rp-aigw-id by placing
it in additional_kwargs (which lands in the request body) and default_headers
(which sets HTTP headers); update the OpenAI client construction inside the
use_gateway branch so rp-aigw-id is only set via default_headers and remove it
from additional_kwargs—locate the OpenAI(...) call (symbol: OpenAI) assigned to
llm in the use_gateway block and delete the rp-aigw-id entry from
additional_kwargs while keeping default_headers={"rp-aigw-id":
os.getenv("REDPANDA_AI_GATEWAY_ID")}.
♻️ Duplicate comments (4)
modules/ai-agents/pages/ai-gateway/observability-logs.adoc (1)
31-731: Resolve remaining PLACEHOLDER sections before publish.
This doc still contains multiple “PLACEHOLDER” markers (UI paths, screenshots, retention/export, integrations, redaction, RBAC, etc.). These are core reference points and should be finalized or explicitly scoped as internal/TBD.modules/ai-agents/pages/ai-gateway/observability-metrics.adoc (1)
31-849: Resolve remaining PLACEHOLDER sections before publish.
This page still has multiple “PLACEHOLDER” markers (UI paths, screenshots, alerting/export details, integrations, retention, delays). These should be finalized or explicitly marked as internal/TBD.modules/ai-agents/pages/ai-gateway/migration-guide.adoc (1)
362-370: Note: Product-specific placeholders remain unresolved.Multiple placeholder sections still need to be filled with concrete values:
- UI navigation paths (line 362, 741)
- Expected gateway overhead/latency (lines 440-444, 770)
- Model catalog links (line 743)
- Log ingestion delay (line 787)
- Response format differences (lines 794-795)
- Streaming support confirmation (line 829)
These were previously flagged in past review comments. Consider either finalizing these values with product/engineering confirmation or marking them as "TBD" with a docs issue tracker.
Also applies to: 440-444, 741-743, 770-770, 787-787, 794-795, 829-829
modules/ai-agents/pages/ai-gateway/mcp-aggregation-guide.adoc (1)
288-310: Note: MCP configuration and operational placeholders remain unresolved.Multiple placeholder sections still need finalization:
- MCP configuration format (UI paths, gateway-level vs per-request deferred loading)
- Orchestrator API security limits (timeout, memory, other constraints)
- MCP server approval workflow details
- MCP server versioning strategy
- MCP-specific metrics availability and dashboard details
- Sandboxing implementation specifics
- LangChain-specific integration code
These were flagged in previous review. Consider finalizing with product/engineering input or marking as "Internal draft - details TBD" until confirmed.
Also applies to: 443-474, 560-621, 664-691, 744-761, 812-821, 1008-1008
🧹 Nitpick comments (1)
modules/ai-agents/pages/ai-gateway/mcp-aggregation-guide.adoc (1)
936-970: Consider enhancing the tool call handling example.The Python integration example shows a basic tool call pattern, but it could be improved for production use:
Incomplete message history (lines 959-970): The follow-up completion after tool execution only includes three messages (user, assistant, tool result). For multi-turn tool usage, you should include the full conversation history or at least maintain context across tool calls.
Missing tools parameter in follow-up call (line 959): The second
client.chat.completions.create()call doesn't include thetoolsparameter, preventing the model from making additional tool calls if needed for multi-step workflows.💡 Enhanced example with proper message history
# Agent uses tools +messages = [{"role": "user", "content": "Query the database for premium users"}] + response = client.chat.completions.create( model="anthropic/claude-sonnet-3.5", - messages=[ - {"role": "user", "content": "Query the database for premium users"} - ], + messages=messages, tools=tools, tool_choice="auto" ) # Handle tool calls if response.choices[0].message.tool_calls: + messages.append(response.choices[0].message) + for tool_call in response.choices[0].message.tool_calls: # Execute tool via gateway tool_result = requests.post( f"https://{CLUSTER_ID}.cloud.redpanda.com/ai-gateway/mcp/tools/{tool_call.function.name}", headers={ "Authorization": f"Bearer {os.getenv('REDPANDA_CLOUD_TOKEN')}", "rp-aigw-id": os.getenv("GATEWAY_ID") }, json=json.loads(tool_call.function.arguments) ) - # Continue conversation with tool result + messages.append({ + "role": "tool", + "tool_call_id": tool_call.id, + "content": json.dumps(tool_result.json()) + }) + + # Continue conversation with full history and tools available - response = client.chat.completions.create( - model="anthropic/claude-sonnet-3.5", - messages=[ - {"role": "user", "content": "Query the database for premium users"}, - response.choices[0].message, - { - "role": "tool", - "tool_call_id": tool_call.id, - "content": json.dumps(tool_result.json()) - } - ] - ) + response = client.chat.completions.create( + model="anthropic/claude-sonnet-3.5", + messages=messages, + tools=tools, # Include tools for potential follow-up calls + tool_choice="auto" + )
modules/ai-agents/pages/ai-gateway/integrations/cline-admin.adoc
Outdated
Show resolved
Hide resolved
modules/ai-agents/pages/ai-gateway/integrations/github-copilot-user.adoc
Outdated
Show resolved
Hide resolved
- Change config file path from ~/.claude/config.json to ~/.claude.json - Add mention of project-level .mcp.json alternative - Replace "transport": "http" with "type": "http" per MCP spec - Remove undocumented apiProviders sections from all examples - Remove request retry configuration section (used apiProviders) - Remove troubleshooting item about missing apiProviders - Update all troubleshooting commands to use correct file path Files updated: - claude-code-admin.adoc - claude-code-user.adoc Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Clarify that Claude Code supports ${VAR} syntax for environment variable
interpolation in the mcpServers section, specifying which variables will
be resolved at runtime (REDPANDA_GATEWAY_URL, REDPANDA_GATEWAY_ID,
REDPANDA_API_KEY).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Remove unofficial VS Code settings keys (cline.apiProvider, cline.apiBaseUrl, cline.customHeaders) and document that API/provider configuration is managed via Cline extension global state, not settings.json. Replace MCP configuration to use cline_mcp_settings.json with official schema: - Change "transport": "http" to "type": "streamableHttp" - Remove "cline.mcpServers" wrapper (use "mcpServers" directly) - Add "Cline > Mcp: Mode" toggle for MCP enablement - Document Cline UI and cline_mcp_settings.json as config methods Update configuration scope section to reflect extension global state storage model instead of VS Code settings.json levels. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Replace "YOUR_API_TOKEN" with "${REDPANDA_API_TOKEN}" environment
variable in the curl Authorization header to avoid secret scanner
false positives.
Add inline comment and NOTE block to inform users they must set the
environment variable before running the command.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Convert configuration from JSON to YAML format: - Change file name from ~/.continue/config.json to ~/.continue/config.yaml - Convert all JSON examples to YAML syntax - Update models, tabAutocompleteModel, apiBase, apiKey references Update MCP configuration to official schema: - Change modelContextProtocolServers (plural) to modelContextProtocolServer (singular) - Replace transport type "http" with "streamable-http" (valid value) - Add recommended directory-based configuration using ~/.continue/mcpServers/ - Provide alternative inline configuration in config.yaml Document that Continue.dev automatically discovers MCP server configurations in the mcpServers/ directory, which is the preferred configuration method. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Replace legacy JSON config with YAML using correct secrets interpolation:
- Change from ${VAR} to ${{ secrets.VAR }} syntax
- Update models -> apiKey, apiBase with secrets.REDPANDA_API_KEY and
secrets.REDPANDA_GATEWAY_URL
- Update requestOptions.headers.rp-aigw-id with secrets.REDPANDA_GATEWAY_ID
- Update experimental.modelContextProtocolServer fields (changed from plural
to singular per schema)
- Change transport type from "http" to "streamable-http"
- Replace transport.url, Authorization header, and rp-aigw-id header with
secrets interpolation
Add IMPORTANT note that ${VAR} syntax from config.json is not supported
and will be treated as literal strings in Continue.dev.
Replace shell environment variable exports with instructions to set
secrets in Continue.dev settings UI.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add IMPORTANT note that VS Code settings.json does not support ${VAR}
interpolation - such placeholders are treated as literal strings.
Provide two alternatives for handling sensitive credentials:
Option 1: Script-based generation
- Add bash script that reads environment variables and generates
settings.json with actual values for cursor.overrideOpenAIBaseUrl,
cursor.overrideOpenAIApiKey, and openai.additionalHeaders
- Add PowerShell script for Windows users with ConvertTo-Json
- Include instructions to run script before launching Cursor
Option 2: Manual value replacement
- Show example with concrete placeholder values users must replace
- Add security note about file permissions (chmod 600)
Remove misleading instructions to set environment variables and launch
from terminal, as this does not enable interpolation in settings.json.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Update Option 1 (built-in custom models): - Change github.copilot.advanced.customModels to correct setting: github.copilot.chat.customOAIModels (experimental chat OAI models) - Restructure configuration as array instead of nested object - Add IMPORTANT note that API keys/headers must be configured via Copilot Chat UI, not settings.json - Replace environment variable instructions with Copilot Chat → Manage Models UI flow: Add Model → Select provider → Enter API key and custom headers (rp-aigw-id) - Remove references to non-working environment variable approach Update Option 2 (OAI Compatible Provider extension): - Replace incorrect oai.provider.* settings with actual extension settings: oaicopilot.baseUrl and oaicopilot.models - Remove non-existent useOAIProvider setting from configuration - Add IMPORTANT note that API keys/headers are NOT set in settings.json - Document correct flow: Copilot Chat → Manage Models → Configure provider → Enter API key and custom headers - Remove environment variable instructions that don't apply - Remove unnecessary window reload step Both options now correctly document that users must use the Copilot Chat → Manage Models UI to configure API keys and custom headers. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Remove rp-aigw-id from additional_kwargs in the LlamaIndex OpenAI client configuration. The header was being set twice - once in additional_kwargs (which goes in the request body) and once in default_headers (which sets HTTP headers). Keep only the default_headers configuration to correctly set the rp-aigw-id as an HTTP header. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
1 similar comment
✅ Actions performedReview triggered.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 3
🤖 Fix all issues with AI agents
In `@modules/ai-agents/pages/ai-gateway/integrations/continue-admin.adoc`:
- Around line 659-663: The doc incorrectly references `config.json` while
Continue.dev uses `config.yaml`; update all occurrences in this section so
examples and warnings refer to `config.yaml` (e.g., change "`config.json`" to
"`config.yaml`" and update the chmod example to `~/.continue/config.yaml`), and
keep the guidance points the same: never commit the config file, restrict
filesystem permissions, and rotate tokens if compromised.
In `@modules/ai-agents/pages/ai-gateway/integrations/continue-user.adoc`:
- Around line 309-324: The example .continuerc.json uses unsupported ${VAR}
interpolation (e.g., "${PROJECT_API_KEY}" and "${PROJECT_GATEWAY_ID}") which
will be treated as literal strings; update the `.continuerc.json` example to use
explicit placeholder text (like PROJECT_API_KEY_PLACEHOLDER) or switch the docs
to show using a supported format (recommend using config.yaml with ${{ secrets.*
}} interpolation or config.ts for programmatic env access) and mention that
config.json does not support ${VAR} expansion to avoid confusion.
- Around line 285-292: Replace the incorrect YAML key
experimental.modelContextProtocolServer with the correct config.yaml schema key
mcpServers and convert the single object into an array entry; update the block
so mcpServers is an array of server objects each containing transport, url and
headers (keeping transport.type, transport.url and headers.Authorization and
headers.rp-aigw-id) to match the structure used for modelContextProtocolServers
in the JSON example; ensure the unique symbols updated are
experimental.modelContextProtocolServer -> mcpServers and transport/url/headers
remain under the array element.
♻️ Duplicate comments (1)
modules/ai-agents/pages/ai-gateway/integrations/github-copilot-user.adoc (1)
274-278: Correct invalid configuration settings—oai.provider.*settings do not exist in the OAI Compatible Provider extension.Multiple sections use
oai.provider.*settings (e.g.,oai.provider.headers,oai.provider.endpoint,oai.provider.apiKey,oai.provider.models,oai.provider.timeout,oai.provider.debug) which do not exist in the OAI Compatible Provider extension's documented API. Based on the past review and official documentation, the extension uses:
oaicopilot.baseUrl(notoai.provider.endpoint)oaicopilot.models(notoai.provider.models)- API keys and custom headers are configured via Copilot Chat → Manage Models UI, not in
settings.jsonThese incorrect settings will cause non-functional configurations for users.
🔧 Required corrections
Lines 274-278, 294-299 (Multi-tenancy, Approach 1):
{ - "oai.provider.headers": { + // Note: Headers must be configured via Copilot Chat → Manage Models UI + // This example shows the structure for documentation purposes only + "oaicopilot.baseUrl": "https://gw.ai.panda.com/v1", + "customHeaders": { "rp-aigw-id": "team-a-gateway-123" } }Lines 367-375 (Centralized configuration):
{ - "oai.provider.endpoint": "https://gw.company.com/v1", - "oai.provider.apiKey": "${env:COPILOT_GATEWAY_KEY}", - "oai.provider.headers": { + "oaicopilot.baseUrl": "https://gw.company.com/v1", + // Note: API keys and headers must be configured via Copilot Chat → Manage Models UI + // Environment variables shown here for reference only + "headers": { "rp-aigw-id": "${env:COPILOT_GATEWAY_ID}", "x-user-email": "${env:USER_EMAIL}", "x-department": "${env:DEPARTMENT}" } }Add a note in each section explaining that while
oaicopilot.baseUrlandoaicopilot.modelscan be set insettings.json, API keys and custom headers must be configured through the Copilot Chat UI (Manage Models).Also applies to: 367-375, 505-522, 674-678, 769-774
🧹 Nitpick comments (6)
modules/ai-agents/pages/ai-gateway/integrations/cursor-user.adoc (1)
546-551: Consider using environment variable syntax in curl examples to avoid triggering security scanners.The placeholder
YOUR_API_KEYin the curl command may trigger secret detection tools. While this is a documentation example, using environment variable syntax (e.g.,${REDPANDA_API_KEY}or$REDPANDA_API_KEY) would better demonstrate secure credential handling and avoid false positives from security scanners.♻️ Suggested improvement
[,bash] ---- -curl -H "Authorization: Bearer YOUR_API_KEY" \ +curl -H "Authorization: Bearer ${REDPANDA_API_KEY}" \ -H "rp-aigw-id: GATEWAY_ID" \ https://gw.ai.panda.com/v1/models ----modules/ai-agents/pages/ai-gateway/integrations/claude-code-user.adoc (1)
307-312: Consider using environment variable syntax in curl examples for consistency.Similar to other integration docs, consider using environment variable syntax (
${REDPANDA_API_KEY}) instead of placeholder text to demonstrate secure credential handling and maintain consistency across the documentation.♻️ Suggested improvement
[,bash] ---- -curl -H "Authorization: Bearer YOUR_API_KEY" \ +curl -H "Authorization: Bearer ${REDPANDA_API_KEY}" \ -H "rp-aigw-id: GATEWAY_ID" \ https://gw.ai.panda.com/mcp ----modules/ai-agents/pages/ai-gateway/integrations/github-copilot-user.adoc (1)
632-637: Use environment variable syntax in curl authentication examples.The placeholder
YOUR_API_KEYmay trigger security scanners. Use environment variable syntax to demonstrate secure credential handling.♻️ Suggested improvement
[,bash] ---- -curl -H "Authorization: Bearer YOUR_API_KEY" \ +curl -H "Authorization: Bearer ${REDPANDA_API_KEY}" \ -H "rp-aigw-id: GATEWAY_ID" \ https://gw.ai.panda.com/v1/models ----Also applies to: 672-678
modules/ai-agents/pages/ai-gateway/integrations/claude-code-admin.adoc (1)
402-413: Consider using environment variable syntax in curl examples.For consistency with security best practices across the documentation, consider using environment variable syntax in the curl example.
♻️ Suggested improvement
[source,bash] ---- +# Set REDPANDA_API_TOKEN environment variable before running curl https://{CLUSTER_ID}.cloud.redpanda.com/api/ai-gateway/logs \ - -H "Authorization: Bearer YOUR_API_TOKEN" \ + -H "Authorization: Bearer ${REDPANDA_API_TOKEN}" \ -H "Content-Type: application/json" \ -d '{ "gateway_id": "GATEWAY_ID", "start_time": "2026-01-01T00:00:00Z", "end_time": "2026-01-14T23:59:59Z", "limit": 100 }' ----modules/ai-agents/pages/ai-gateway/integrations/continue-admin.adoc (1)
608-617: Use environment variable syntax in curl example for security best practices.For consistency with other admin documentation and to avoid triggering security scanners, use environment variable syntax in the curl authentication header.
♻️ Suggested improvement
[source,bash] ---- +# Set REDPANDA_API_TOKEN environment variable before running curl https://{CLUSTER_ID}.cloud.redpanda.com/api/ai-gateway/logs \ - -H "Authorization: Bearer YOUR_API_TOKEN" \ + -H "Authorization: Bearer ${REDPANDA_API_TOKEN}" \ -H "Content-Type: application/json" \ -d '{ "gateway_id": "GATEWAY_ID", "start_time": "2026-01-01T00:00:00Z", "end_time": "2026-01-14T23:59:59Z", "limit": 100 }' ----modules/ai-agents/pages/ai-gateway/integrations/continue-user.adoc (1)
592-596: Prefer env var in curl example to avoid shell-history leaks.Even with placeholders, examples like this are commonly copied verbatim. Consider using an env var in the snippet.
✅ Suggested tweak
-curl -H "Authorization: Bearer YOUR_API_KEY" \ +curl -H "Authorization: Bearer ${REDPANDA_API_KEY}" \ -H "rp-aigw-id: GATEWAY_ID" \ https://gw.ai.panda.com/v1/models
modules/ai-agents/pages/ai-gateway/integrations/continue-admin.adoc
Outdated
Show resolved
Hide resolved
modules/ai-agents/pages/ai-gateway/integrations/continue-user.adoc
Outdated
Show resolved
Hide resolved
Update security guidance section to reference config.yaml instead of config.json: - Change "stores the API token in plain text in config.json" to config.yaml - Update version control warning to reference config.yaml - Update chmod example from ~/.continue/config.json to ~/.continue/config.yaml Keep guidance points the same: never commit the config file, restrict filesystem permissions, and rotate tokens if compromised. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Update Continue.dev documentation to use the correct MCP server configuration schema: - Change from experimental.modelContextProtocolServer (singular) to mcpServers (plural) at root level - Add array syntax with dash (-) before transport configuration - Update continue-user.adoc inline config.yaml examples (2 instances) - Update continue-admin.adoc inline config.yaml example - Fix troubleshooting references to use new mcpServers schema - Convert JSON config example to YAML with new schema for consistency The new schema allows multiple MCP servers to be configured and aligns with Continue.dev's current config.yaml structure. Directory-based configuration in ~/.continue/mcpServers/ already uses correct schema. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Replace misleading ${VAR} syntax in .continuerc.json example with
explicit placeholder text (your_project_api_key_here, etc.) to avoid
confusion.
Add IMPORTANT note explaining that .continuerc.json does not support
environment variable interpolation and must use hardcoded values.
Direct users to config.yaml with ${{ secrets.* }} syntax or config.ts
for programmatic environment access as alternatives.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Address critical documentation issues identified in audit:
1. Remove checkboxes from learning objectives in both user and admin
guides per content architecture standards
2. Add config file format clarification section explaining JSON vs YAML,
when each is required/recommended, helping users understand which
format to choose
3. Fix environment variable syntax documentation:
- Add explicit anchor [[configure-env-vars]] for cross-references
- Clarify ${{ secrets.* }} is YAML-only
- Explain config.json does not support any variable interpolation
- Clarify ${VAR} syntax is treated as literal string
4. Add security warning in admin guide about hardcoded tokens with
cross-reference to environment variable interpolation best practices
5. Update admin guide to acknowledge both JSON and YAML formats are
supported, explaining why YAML is recommended for MCP features
These changes eliminate confusion about config file formats and improve
security guidance for production deployments.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
…egration files Apply content architecture standards to all AI Gateway integration documentation: - Remove [ ] checkbox formatting from learning objectives - Remove trailing periods from objective fragments Files updated: - cursor-user.adoc, cursor-admin.adoc - github-copilot-user.adoc, github-copilot-admin.adoc - claude-code-user.adoc, claude-code-admin.adoc - cline-user.adoc, cline-admin.adoc Learning objectives should be simple list items without interactive elements or punctuation for fragments. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
… all integration files" This reverts commit f754065. Checkboxes [ ] ARE the doc team standard for learning objectives in this repository, as evidenced by the MCP documentation pages and other content. The audit agent was incorrect in recommending their removal. Restored checkboxes and periods to learning objectives in all integration files to match the established pattern: * [ ] Learning objective text. Examples of this pattern in the repo: - modules/ai-agents/pages/mcp/remote/quickstart.adoc - modules/ai-agents/pages/mcp/remote/concepts.adoc - modules/ai-agents/pages/mcp/local/configuration.adoc
Description
This pull request introduces the AI Gateway in Redpanda Cloud.
Resolves https://redpandadata.atlassian.net/browse/DOC-1867
Review deadline: Feb 6
Page previews
Checks