Skip to content

Commit ff4c33e

Browse files
committed
feat(conformance): add client scenarios for elicitation and SSE retry
Implement remaining client conformance scenarios (excluding auth): - elicitation-sep1034-client-defaults: Validates default value application from JSON schemas (SEP-1034) - sse-retry: Tests SSE reconnection behavior (SEP-1699) Added createClientWithElicitation() method with elicitation handler that automatically applies defaults from schema properties. Handler supports string, integer, number, enum, and boolean default values. Test results for elicitation scenario: 5/5 checks passing - String defaults applied correctly - Integer defaults applied correctly - Number defaults applied correctly - Enum defaults applied correctly - Boolean defaults applied correctly Updated documentation with scenario descriptions and usage examples.
1 parent b869556 commit ff4c33e

File tree

3 files changed

+302
-11
lines changed

3 files changed

+302
-11
lines changed

conformance-tests/VALIDATION_RESULTS.md

Lines changed: 140 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,98 @@
22

33
## Summary
44

5-
The Java SDK conformance server implementation has been validated against the official MCP conformance test suite. Out of 40 total test checks in the "active" suite, **36 passed (90%)** and **4 failed (10%)**.
5+
The Java SDK has been validated against the official MCP conformance test suite for both server and client implementations.
6+
7+
### Server Tests
8+
Out of 40 total test checks in the "active" suite, **36 passed (90%)** and **4 failed (10%)**.
9+
10+
### Client Tests
11+
The client conformance implementation supports 4 core scenarios (excluding auth):
12+
- ✅ initialize
13+
- ✅ tools_call
14+
- ✅ elicitation-sep1034-client-defaults
15+
- ✅ sse-retry
16+
17+
## Client Test Results
18+
19+
### ✅ Implemented Client Scenarios (4/4)
20+
21+
#### 1. initialize
22+
**Status:** ✅ Implemented
23+
**Description:** Tests the MCP client initialization handshake
24+
**Validates:**
25+
- Protocol version negotiation
26+
- Client info (name and version)
27+
- Server capabilities handling
28+
- Proper connection establishment and closure
29+
30+
#### 2. tools_call
31+
**Status:** ✅ Implemented
32+
**Description:** Tests tool discovery and invocation
33+
**Validates:**
34+
- Client initialization
35+
- Listing available tools from server
36+
- Calling the `add_numbers` tool with arguments (a=5, b=3)
37+
- Processing tool results
38+
39+
#### 3. elicitation-sep1034-client-defaults
40+
**Status:** ✅ Implemented
41+
**Description:** Tests that client applies default values for omitted elicitation fields (SEP-1034)
42+
**Validates:**
43+
- Client properly applies default values from JSON schema
44+
- Supports string, integer, number, enum, and boolean defaults
45+
- Correctly handles elicitation requests from server
46+
- Sends complete responses with all required fields
47+
48+
#### 4. sse-retry
49+
**Status:** ✅ Implemented
50+
**Description:** Tests client respects SSE retry field timing and reconnects properly (SEP-1699)
51+
**Validates:**
52+
- Client reconnects after SSE stream closure
53+
- Respects the retry field timing (waits specified milliseconds)
54+
- Sends Last-Event-ID header on reconnection
55+
- Handles graceful stream closure as reconnectable
56+
57+
### Client Implementation Details
58+
59+
The client conformance tests use:
60+
- **Transport:** `HttpClientStreamableHttpTransport` (JDK HTTP Client)
61+
- **Client Type:** `McpAsyncClient` with reactive (Reactor) API
62+
- **Configuration:** 30-second request timeout, test-client/1.0.0 identification
63+
- **Protocol:** Latest Streamable HTTP protocol (2025-03-26)
64+
65+
### Running Client Tests
66+
67+
Build the executable JAR:
68+
```bash
69+
cd conformance-tests/client-jdk-http-client
70+
../../mvnw clean package -DskipTests
71+
```
72+
73+
Run with conformance framework:
74+
```bash
75+
npx @modelcontextprotocol/conformance client \
76+
--command "java -jar conformance-tests/client-jdk-http-client/target/client-jdk-http-client-0.18.0-SNAPSHOT.jar" \
77+
--scenario initialize
78+
79+
npx @modelcontextprotocol/conformance client \
80+
--command "java -jar conformance-tests/client-jdk-http-client/target/client-jdk-http-client-0.18.0-SNAPSHOT.jar" \
81+
--scenario tools_call
682

7-
## Test Results
83+
npx @modelcontextprotocol/conformance client \
84+
--command "java -jar conformance-tests/client-jdk-http-client/target/client-jdk-http-client-0.18.0-SNAPSHOT.jar" \
85+
--scenario elicitation-sep1034-client-defaults
86+
87+
npx @modelcontextprotocol/conformance client \
88+
--command "java -jar conformance-tests/client-jdk-http-client/target/client-jdk-http-client-0.18.0-SNAPSHOT.jar" \
89+
--scenario sse-retry
90+
```
91+
92+
### Excluded Scenarios
93+
94+
**Auth Scenarios:** Authentication-related scenarios were excluded as per requirements. The conformance framework includes 15+ auth scenarios that test OAuth2, OIDC, and various authentication flows. These can be added in future iterations.
95+
96+
## Server Test Results
897

998
### ✅ Passing Tests (36/40)
1099

@@ -91,12 +180,30 @@ The Java SDK conformance server implementation has been validated against the of
91180

92181
## Changes Made
93182

94-
### 1. Added Completion Support
183+
### Client Conformance Implementation
184+
185+
#### 1. Base Client Scenarios
186+
- Implemented `initialize` scenario for basic handshake testing
187+
- Implemented `tools_call` scenario for tool discovery and invocation
188+
189+
#### 2. Elicitation Defaults (SEP-1034)
190+
- Implemented `elicitation-sep1034-client-defaults` scenario
191+
- Tests client properly applies default values from JSON schema
192+
- Validates all primitive types: string, integer, number, enum, boolean
193+
194+
#### 3. SSE Retry Handling (SEP-1699)
195+
- Implemented `sse-retry` scenario
196+
- Tests client respects retry field timing
197+
- Validates graceful reconnection with Last-Event-ID header
198+
199+
### Server Conformance Implementation
200+
201+
#### 1. Added Completion Support
95202
- Enabled `completions` capability in server capabilities
96203
- Implemented completion handler for `test_prompt_with_arguments` prompt
97204
- Returns minimal completion with required `total` field set to 0
98205

99-
### 2. Added SEP-1034 Elicitation Defaults Tool
206+
#### 2. Added SEP-1034 Elicitation Defaults Tool
100207
- Implemented `test_elicitation_sep1034_defaults` tool
101208
- Supports default values for all primitive types:
102209
- String: "John Doe"
@@ -105,7 +212,7 @@ The Java SDK conformance server implementation has been validated against the of
105212
- Enum: "active" (from ["active", "inactive", "pending"])
106213
- Boolean: true
107214

108-
### 3. Added SEP-1330 Enum Schema Improvements Tool
215+
#### 3. Added SEP-1330 Enum Schema Improvements Tool
109216
- Implemented `test_elicitation_sep1330_enums` tool
110217
- Supports all 5 enum variants:
111218
- Untitled single-select (enum array)
@@ -114,7 +221,7 @@ The Java SDK conformance server implementation has been validated against the of
114221
- Untitled multi-select (array with items.enum)
115222
- Titled multi-select (array with items.anyOf)
116223

117-
### 4. Enabled Resources Capability
224+
#### 4. Enabled Resources Capability
118225
- Added `resources(true, false)` to server capabilities
119226
- Enables subscribe capability (though not fully implemented in SDK)
120227

@@ -155,7 +262,9 @@ The HTTP transport does not validate Host/Origin headers, making localhost serve
155262

156263
## Testing Instructions
157264

158-
To reproduce these tests:
265+
### Server Tests
266+
267+
To reproduce server tests:
159268

160269
```bash
161270
# Start the conformance server
@@ -166,12 +275,35 @@ cd conformance-tests/server-servlet
166275
npx @modelcontextprotocol/conformance server --url http://localhost:8080/mcp --suite active
167276
```
168277

169-
To test individual scenarios:
278+
To test individual server scenarios:
170279

171280
```bash
172281
npx @modelcontextprotocol/conformance server --url http://localhost:8080/mcp --scenario tools-call-with-progress --verbose
173282
```
174283

284+
### Client Tests
285+
286+
To test client scenarios:
287+
288+
```bash
289+
# Build the client JAR first
290+
cd conformance-tests/client-jdk-http-client
291+
../../mvnw clean package -DskipTests
292+
293+
# Run individual scenarios
294+
npx @modelcontextprotocol/conformance client \
295+
--command "java -jar target/client-jdk-http-client-0.18.0-SNAPSHOT.jar" \
296+
--scenario initialize \
297+
--verbose
298+
299+
# Test all client scenarios
300+
for scenario in initialize tools_call elicitation-sep1034-client-defaults sse-retry; do
301+
npx @modelcontextprotocol/conformance client \
302+
--command "java -jar target/client-jdk-http-client-0.18.0-SNAPSHOT.jar" \
303+
--scenario $scenario
304+
done
305+
```
306+
175307
## Conclusion
176308

177309
The Java SDK conformance implementation demonstrates strong compatibility with the MCP specification, achieving 90% test pass rate. The failing tests represent known limitations that require SDK-level fixes rather than implementation issues in the conformance server itself.

conformance-tests/client-jdk-http-client/README.md

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,18 @@ Currently implemented scenarios:
2929
- Calls the `add_numbers` tool with test arguments (a=5, b=3)
3030
- Validates the tool result
3131

32+
- **elicitation-sep1034-client-defaults**: Tests client applies default values for omitted elicitation fields (SEP-1034)
33+
- Initializes the client
34+
- Lists available tools from the server
35+
- Calls the `test_client_elicitation_defaults` tool
36+
- Validates that the client properly applies default values from JSON schema to elicitation responses
37+
38+
- **sse-retry**: Tests client respects SSE retry field timing and reconnects properly (SEP-1699)
39+
- Initializes the client
40+
- Lists available tools from the server
41+
- Calls the `test_reconnection` tool which triggers SSE stream closure
42+
- Validates that the client properly reconnects with correct timing and Last-Event-ID header
43+
3244
## Building
3345

3446
Build the executable JAR:
@@ -57,6 +69,14 @@ npx @modelcontextprotocol/conformance client \
5769
npx @modelcontextprotocol/conformance client \
5870
--command "java -jar conformance-tests/client-jdk-http-client/target/client-jdk-http-client-0.18.0-SNAPSHOT.jar" \
5971
--scenario tools_call
72+
73+
npx @modelcontextprotocol/conformance client \
74+
--command "java -jar conformance-tests/client-jdk-http-client/target/client-jdk-http-client-0.18.0-SNAPSHOT.jar" \
75+
--scenario elicitation-sep1034-client-defaults
76+
77+
npx @modelcontextprotocol/conformance client \
78+
--command "java -jar conformance-tests/client-jdk-http-client/target/client-jdk-http-client-0.18.0-SNAPSHOT.jar" \
79+
--scenario sse-retry
6080
```
6181

6282
Run with verbose output:
@@ -139,7 +159,7 @@ case "new-scenario":
139159

140160
Future enhancements:
141161

142-
- Add more scenarios (auth, elicitation, etc.)
162+
- Add auth scenarios (currently excluded as per requirements)
143163
- Implement a comprehensive "everything-client" pattern
144164
- Add to CI/CD pipeline
145165
- Create expected-failures baseline for known issues

0 commit comments

Comments
 (0)