Skip to content

Commit 7a9131f

Browse files
rdimitrovclaude
rdimitrov
and
claude
committed
Update docs for ToolHive v0.12.3–v0.13.0
Catch up documentation with features shipped in v0.12.3 through v0.13.0. Auto-generated CLI/CRD reference docs were already current; these changes cover manual doc updates verified against source code at each release tag. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 0a956b7 commit 7a9131f

7 files changed

Lines changed: 222 additions & 43 deletions

File tree

‎docs/toolhive/concepts/backend-auth.mdx‎

Lines changed: 18 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -211,18 +211,25 @@ deployments using the ToolHive Operator.
211211
- **Direct upstream redirect:** The embedded authorization server redirects
212212
clients directly to the upstream provider for authentication (for example,
213213
GitHub or Atlassian).
214-
- **Single upstream provider:** Currently supports one upstream identity
215-
provider per configuration.
216-
217-
:::info[Chained authentication not yet supported]
218-
219-
The embedded authorization server redirects clients directly to the upstream
220-
provider. This means the upstream provider must be the service whose API the MCP
221-
server calls. Chained authentication—where a client authenticates with a
214+
- **Multiple upstream providers (VirtualMCPServer):** VirtualMCPServer supports
215+
configuring multiple upstream identity providers with sequential
216+
authentication. When multiple providers are configured, the authorization
217+
server chains the authentication flow through each provider in sequence,
218+
collecting tokens from all of them. This enables scenarios where backend tools
219+
require tokens from different providers (such as a corporate IdP and GitHub).
220+
MCPServer and MCPRemoteProxy support a single upstream provider per
221+
configuration.
222+
223+
:::info[Chained authentication for MCPServer]
224+
225+
MCPServer and MCPRemoteProxy support only one upstream provider. The embedded
226+
authorization server redirects clients directly to that provider, so the
227+
provider must be the service whose API the MCP server calls. If your MCPServer
228+
deployment requires chained authentication—where a client authenticates with a
222229
corporate IdP like Okta, which then federates to an external provider like
223-
GitHub—is not yet supported. If your deployment requires this pattern, consider
224-
using [token exchange](#same-idp-with-token-exchange) with a federated identity
225-
provider instead.
230+
GitHub—consider using [token exchange](#same-idp-with-token-exchange) with a
231+
federated identity provider instead, or use a VirtualMCPServer with multiple
232+
upstream providers.
226233

227234
:::
228235

‎docs/toolhive/guides-k8s/auth-k8s.mdx‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -464,13 +464,13 @@ kubectl apply -f embedded-auth-config.yaml
464464

465465
**Configuration reference:**
466466

467-
| Field | Description |
468-
| ---------------------- | ---------------------------------------------------------------------------------------------------------------------- |
469-
| `issuer` | HTTPS URL identifying this authorization server. Appears in the `iss` claim of issued JWTs. |
470-
| `signingKeySecretRefs` | References to Secrets containing JWT signing keys. First key is active; additional keys support rotation. |
471-
| `hmacSecretRefs` | References to Secrets with symmetric keys for signing authorization codes and refresh tokens. |
472-
| `tokenLifespans` | Configurable durations for access tokens (default: 1h), refresh tokens (default: 168h), and auth codes (default: 10m). |
473-
| `upstreamProviders` | Configuration for the upstream identity provider. Currently supports one provider. |
467+
| Field | Description |
468+
| ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
469+
| `issuer` | HTTPS URL identifying this authorization server. Appears in the `iss` claim of issued JWTs. |
470+
| `signingKeySecretRefs` | References to Secrets containing JWT signing keys. First key is active; additional keys support rotation. |
471+
| `hmacSecretRefs` | References to Secrets with symmetric keys for signing authorization codes and refresh tokens. |
472+
| `tokenLifespans` | Configurable durations for access tokens (default: 1h), refresh tokens (default: 168h), and auth codes (default: 10m). |
473+
| `upstreamProviders` | Configuration for upstream identity providers. MCPServer and MCPRemoteProxy support one provider; VirtualMCPServer supports multiple providers for sequential authentication. |
474474

475475
**Step 5: Create the MCPServer resource**
476476

‎docs/toolhive/guides-k8s/redis-session-storage.mdx‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
title: Redis Sentinel session storage
33
description:
44
How to deploy Redis Sentinel and configure persistent session storage for the
5-
ToolHive embedded authorization server.
5+
ToolHive embedded authorization server and horizontal scaling.
66
---
77

88
Deploy Redis Sentinel and configure it as the session storage backend for the
@@ -12,6 +12,11 @@ re-authenticate. Redis Sentinel provides persistent storage with automatic
1212
master discovery, ACL-based access control, and optional failover when replicas
1313
are configured.
1414

15+
Redis session storage is also required for
16+
[horizontal scaling](../guides-vmcp/scaling-and-performance.mdx#session-storage-for-multi-replica-deployments)
17+
when running multiple MCPServer or VirtualMCPServer replicas, so that sessions
18+
are shared across pods.
19+
1520
:::info[Prerequisites]
1621

1722
Before you begin, ensure you have:

‎docs/toolhive/guides-k8s/run-mcp-k8s.mdx‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -455,6 +455,8 @@ kubectl -n <NAMESPACE> describe mcpserver <NAME>
455455

456456
- [Kubernetes CRD reference](../reference/crd-spec.md#apiv1alpha1mcpserver) -
457457
Reference for the `MCPServer` Custom Resource Definition (CRD)
458+
- [Scaling and performance](../guides-vmcp/scaling-and-performance.mdx#mcpserver-horizontal-scaling) -
459+
Configure horizontal scaling with `replicas` and `backendReplicas`
458460
- [Deploy the operator](./deploy-operator.mdx) - Install the ToolHive operator
459461
- [Build MCP containers](../guides-cli/build-containers.mdx) - Create custom MCP
460462
server container images

‎docs/toolhive/guides-vmcp/composite-tools.mdx‎

Lines changed: 68 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ backend MCP servers, handling dependencies and collecting results.
1919
wait for their prerequisites
2020
- **Template expansion**: Dynamic arguments using step outputs
2121
- **Elicitation**: Request user input mid-workflow (approval gates, choices)
22+
- **Iteration**: Loop over collections with forEach steps
2223
- **Error handling**: Configurable abort, continue, or retry behavior
2324
- **Timeouts**: Workflow and per-step timeout configuration
2425

@@ -290,7 +291,7 @@ spec:
290291

291292
### Steps
292293

293-
Each step can be a tool call or an elicitation:
294+
Each step can be a tool call, an elicitation, or a forEach loop:
294295

295296
```yaml title="VirtualMCPServer resource"
296297
spec:
@@ -344,6 +345,62 @@ spec:
344345
timeout: '5m'
345346
```
346347

348+
### forEach steps
349+
350+
Iterate over a collection from a previous step's output and execute a tool call
351+
for each item:
352+
353+
```yaml title="VirtualMCPServer resource"
354+
spec:
355+
config:
356+
compositeTools:
357+
- name: scan_repositories
358+
description: Check each repository for security advisories
359+
parameters:
360+
type: object
361+
properties:
362+
org:
363+
type: string
364+
required:
365+
- org
366+
steps:
367+
- id: list_repos
368+
tool: github_list_repos
369+
arguments:
370+
org: '{{.params.org}}'
371+
# highlight-start
372+
- id: check_advisories
373+
type: forEach
374+
collection: '{{json .steps.list_repos.output.repositories}}'
375+
itemVar: repo
376+
maxParallel: 5
377+
step:
378+
type: tool
379+
tool: github_list_security_advisories
380+
arguments:
381+
repo: '{{.forEach.repo.name}}'
382+
onError:
383+
action: continue
384+
dependsOn: [list_repos]
385+
# highlight-end
386+
```
387+
388+
**forEach fields:**
389+
390+
| Field | Description | Default |
391+
| --------------- | ----------------------------------------------------- | ------- |
392+
| `collection` | Template expression that produces an array | — |
393+
| `itemVar` | Variable name for the current item | — |
394+
| `maxParallel` | Maximum concurrent iterations (max 50) | 10 |
395+
| `maxIterations` | Maximum total iterations (max 1000) | 100 |
396+
| `step` | Inner step definition (tool call to execute per item) | — |
397+
| `onError` | Error handling: `abort` (stop) or `continue` (skip) | abort |
398+
399+
Access the current item inside the inner step using
400+
`{{.forEach.<itemVar>.<field>}}`. In the example above, `{{.forEach.repo.name}}`
401+
accesses the `name` field of the current repository. You can also use
402+
`{{.forEach.index}}` to access the zero-based iteration index.
403+
347404
### Error handling
348405

349406
Configure behavior when steps fail:
@@ -507,13 +564,16 @@ without defaultResults defined
507564

508565
Access workflow context in arguments:
509566

510-
| Template | Description |
511-
| --------------------------- | ------------------------------------------ |
512-
| `{{.params.name}}` | Input parameter |
513-
| `{{.steps.id.output}}` | Step output (map) |
514-
| `{{.steps.id.output.text}}` | Text content from step output |
515-
| `{{.steps.id.content}}` | Elicitation response content |
516-
| `{{.steps.id.action}}` | Elicitation action (accept/decline/cancel) |
567+
| Template | Description |
568+
| -------------------------------- | ------------------------------------------ |
569+
| `{{.params.name}}` | Input parameter |
570+
| `{{.steps.id.output}}` | Step output (map) |
571+
| `{{.steps.id.output.text}}` | Text content from step output |
572+
| `{{.steps.id.content}}` | Elicitation response content |
573+
| `{{.steps.id.action}}` | Elicitation action (accept/decline/cancel) |
574+
| `{{.forEach.<itemVar>}}` | Current forEach item |
575+
| `{{.forEach.<itemVar>.<field>}}` | Field on current forEach item |
576+
| `{{.forEach.index}}` | Zero-based iteration index |
517577

518578
### Template functions
519579

‎docs/toolhive/guides-vmcp/scaling-and-performance.mdx‎

Lines changed: 85 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,12 @@
11
---
22
title: Scaling and Performance
33
description:
4-
How to scale Virtual MCP Server deployments vertically and horizontally.
4+
How to scale MCPServer and Virtual MCP Server deployments vertically and
5+
horizontally.
56
---
67

7-
This guide explains how to scale Virtual MCP Server (vMCP) deployments.
8+
This guide explains how to scale MCPServer and Virtual MCP Server (vMCP)
9+
deployments.
810

911
## Vertical scaling
1012

@@ -37,24 +39,91 @@ higher request volumes.
3739
3840
### How to scale horizontally
3941
40-
The VirtualMCPServer CRD does not have a `replicas` field. The operator creates
41-
a Deployment named `vmcp-<NAME>` (where `<NAME>` is your VirtualMCPServer name)
42-
with 1 replica and preserves the replicas count, allowing you to manage scaling
43-
separately.
42+
Set the `replicas` field in your VirtualMCPServer spec to control the number of
43+
vMCP pods:
44+
45+
```yaml title="VirtualMCPServer resource"
46+
spec:
47+
replicas: 3
48+
```
49+
50+
When `replicas` is not set, the operator does not manage the replica count,
51+
leaving it to an HPA or other external controller. You can also scale manually
52+
or with an HPA:
4453

4554
**Option 1: Manual scaling**
4655

4756
```bash
48-
kubectl scale deployment vmcp-<vmcp-name> -n <NAMESPACE> --replicas=3
57+
kubectl scale deployment vmcp-<VMCP_NAME> -n <NAMESPACE> --replicas=3
4958
```
5059

5160
**Option 2: Autoscaling with HPA**
5261

5362
```bash
54-
kubectl autoscale deployment vmcp-<vmcp-name> -n <NAMESPACE> \
63+
kubectl autoscale deployment vmcp-<VMCP_NAME> -n <NAMESPACE> \
5564
--min=2 --max=5 --cpu-percent=70
5665
```
5766

67+
### Session storage for multi-replica deployments
68+
69+
When running multiple replicas, configure Redis session storage so that sessions
70+
are shared across pods. Without session storage, a request routed to a different
71+
replica than the one that established the session will fail.
72+
73+
```yaml title="VirtualMCPServer resource"
74+
spec:
75+
replicas: 3
76+
sessionStorage:
77+
provider: redis
78+
address: redis-master.toolhive-system.svc.cluster.local:6379
79+
db: 0
80+
keyPrefix: vmcp-sessions
81+
passwordRef:
82+
name: redis-secret
83+
key: password
84+
```
85+
86+
See [Redis Sentinel session storage](../guides-k8s/redis-session-storage.mdx)
87+
for a complete Redis deployment guide.
88+
89+
:::warning
90+
91+
If you configure multiple replicas without session storage, the operator sets a
92+
`SessionStorageMissingForReplicas` status condition on the resource. Ensure
93+
Redis is available before scaling beyond a single replica.
94+
95+
:::
96+
97+
### MCPServer horizontal scaling
98+
99+
MCPServer creates two separate Deployments: one for the proxy runner and one for
100+
the MCP server backend. You can scale each independently:
101+
102+
- `spec.replicas` controls the proxy runner pod count
103+
- `spec.backendReplicas` controls the backend MCP server pod count
104+
105+
```yaml title="MCPServer resource"
106+
spec:
107+
replicas: 2
108+
backendReplicas: 3
109+
sessionStorage:
110+
provider: redis
111+
address: redis-master.toolhive-system.svc.cluster.local:6379
112+
db: 0
113+
keyPrefix: mcp-sessions
114+
passwordRef:
115+
name: redis-secret
116+
key: password
117+
```
118+
119+
:::warning[Stdio transport limitation]
120+
121+
Backends using the `stdio` transport are limited to a single replica. The
122+
operator rejects configurations with `backendReplicas` greater than 1 for stdio
123+
backends.
124+
125+
:::
126+
58127
### When horizontal scaling is challenging
59128

60129
Horizontal scaling works well for **stateless backends** (fetch, search,
@@ -63,22 +132,22 @@ read-only operations) where sessions can be resumed on any instance.
63132
However, **stateful backends** make horizontal scaling difficult:
64133

65134
- **Stateful backends** (Playwright browser sessions, database connections, file
66-
system operations) require requests to be routed to the same vMCP instance
67-
that established the session
135+
system operations) require requests to be routed to the same instance that
136+
established the session
68137
- Session resumption may not work reliably for stateful backends
69138

70-
The `VirtualMCPServer` CRD includes a `sessionAffinity` field that controls how
71-
the Kubernetes Service routes repeated client connections. By default, it uses
72-
`ClientIP` affinity, which routes connections from the same client IP to the
73-
same pod. You can configure this using the `sessionAffinity` field:
139+
The `VirtualMCPServer` and `MCPServer` CRDs include a `sessionAffinity` field
140+
that controls how the Kubernetes Service routes repeated client connections. By
141+
default, it uses `ClientIP` affinity, which routes connections from the same
142+
client IP to the same pod:
74143

75144
```yaml
76145
spec:
77146
sessionAffinity: ClientIP # default
78147
```
79148

80-
For stateful backends, vertical scaling or dedicated vMCP instances per team/use
81-
case are recommended instead of horizontal scaling.
149+
For stateful backends, vertical scaling or dedicated instances per team/use case
150+
are recommended instead of horizontal scaling.
82151

83152
## Next steps
84153

‎docs/toolhive/guides-vmcp/tool-aggregation.mdx‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -146,6 +146,42 @@ spec:
146146
description: 'Create a new GitHub issue in the repository'
147147
```
148148

149+
### Annotation overrides
150+
151+
Override MCP tool annotations to provide hints to LLM clients about tool
152+
behavior. Annotations are optional—only set the fields you want to override:
153+
154+
```yaml title="VirtualMCPServer resource"
155+
spec:
156+
config:
157+
aggregation:
158+
tools:
159+
- workload: github
160+
overrides:
161+
delete_repository:
162+
annotations:
163+
destructiveHint: true
164+
readOnlyHint: false
165+
list_issues:
166+
annotations:
167+
title: 'List GitHub Issues'
168+
readOnlyHint: true
169+
idempotentHint: true
170+
```
171+
172+
**Available annotation fields:**
173+
174+
| Field | Type | Description |
175+
| ----------------- | ------- | -------------------------------------------------- |
176+
| `title` | string | Display title for the tool in MCP clients |
177+
| `readOnlyHint` | boolean | Indicates the tool does not modify data |
178+
| `destructiveHint` | boolean | Indicates the tool may delete or overwrite data |
179+
| `idempotentHint` | boolean | Indicates repeated calls produce the same result |
180+
| `openWorldHint` | boolean | Indicates the tool interacts with external systems |
181+
182+
Annotation overrides can be combined with name and description overrides on the
183+
same tool.
184+
149185
:::info
150186

151187
You can also reference an `MCPToolConfig` resource using `toolConfigRef` instead

0 commit comments

Comments
 (0)