Skip to content

Commit b869556

Browse files
chemicLchemicL
committed
feat(conformance): implement client test suite with initialize and tools_call scenarios
Add JDK HTTP client conformance tests using Streamable HTTP transport. Implements two scenarios: initialize (handshake only) and tools_call (list and invoke add_numbers tool). Both scenarios pass conformance tests. Signed-off-by: Dariusz Jędrzejczyk <dariusz.jedrzejczyk@broadcom.com>
1 parent ba3a5bc commit b869556

5 files changed

Lines changed: 351 additions & 1 deletion

File tree

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ build/
1010
out
1111
/.gradletasknamecache
1212
**/*.flattened-pom.xml
13+
**/dependency-reduced-pom.xml
1314

1415
### IDE - Eclipse/STS ###
1516
.apt_generated

‎conformance-tests/client-jdk-http-client/README.md‎

Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,145 @@
1+
# MCP Conformance Tests - JDK HTTP Client
2+
3+
This module provides a conformance test client implementation for the Java MCP SDK using the JDK HTTP Client with Streamable HTTP transport.
4+
5+
## Overview
6+
7+
The conformance test client is designed to work with the [MCP Conformance Test Framework](https://github.com/modelcontextprotocol/conformance). It validates that the Java MCP SDK client properly implements the MCP specification.
8+
9+
## Architecture
10+
11+
The client reads test scenarios from environment variables and accepts the server URL as a command-line argument, following the conformance framework's conventions:
12+
13+
- **MCP_CONFORMANCE_SCENARIO**: Environment variable specifying which test scenario to run
14+
- **Server URL**: Passed as the last command-line argument
15+
16+
## Supported Scenarios
17+
18+
Currently implemented scenarios:
19+
20+
- **initialize**: Tests the MCP client initialization handshake only
21+
- Validates protocol version negotiation
22+
- Validates clientInfo (name and version)
23+
- Validates proper handling of server capabilities
24+
- Does NOT call any tools or perform additional operations
25+
26+
- **tools_call**: Tests tool discovery and invocation
27+
- Initializes the client
28+
- Lists available tools from the server
29+
- Calls the `add_numbers` tool with test arguments (a=5, b=3)
30+
- Validates the tool result
31+
32+
## Building
33+
34+
Build the executable JAR:
35+
36+
```bash
37+
cd conformance-tests/client-jdk-http-client
38+
../../mvnw clean package -DskipTests
39+
```
40+
41+
This creates an executable JAR at:
42+
```
43+
target/client-jdk-http-client-0.18.0-SNAPSHOT.jar
44+
```
45+
46+
## Running Tests
47+
48+
### Using the Conformance Framework
49+
50+
Run a single scenario:
51+
52+
```bash
53+
npx @modelcontextprotocol/conformance client \
54+
--command "java -jar conformance-tests/client-jdk-http-client/target/client-jdk-http-client-0.18.0-SNAPSHOT.jar" \
55+
--scenario initialize
56+
57+
npx @modelcontextprotocol/conformance client \
58+
--command "java -jar conformance-tests/client-jdk-http-client/target/client-jdk-http-client-0.18.0-SNAPSHOT.jar" \
59+
--scenario tools_call
60+
```
61+
62+
Run with verbose output:
63+
64+
```bash
65+
npx @modelcontextprotocol/conformance client \
66+
--command "java -jar conformance-tests/client-jdk-http-client/target/client-jdk-http-client-0.18.0-SNAPSHOT.jar" \
67+
--scenario initialize \
68+
--verbose
69+
```
70+
71+
### Manual Testing
72+
73+
You can also run the client manually if you have a test server:
74+
75+
```bash
76+
export MCP_CONFORMANCE_SCENARIO=initialize
77+
java -jar conformance-tests/client-jdk-http-client/target/client-jdk-http-client-0.18.0-SNAPSHOT.jar http://localhost:3000/mcp
78+
```
79+
80+
## Test Results
81+
82+
The conformance framework generates test results in `results/initialize-<timestamp>/`:
83+
84+
- `checks.json`: Array of conformance check results with pass/fail status
85+
- `stdout.txt`: Client stdout output
86+
- `stderr.txt`: Client stderr output
87+
88+
## Implementation Details
89+
90+
### Scenario Separation
91+
92+
The implementation follows a clean separation of concerns:
93+
94+
- **initialize scenario**: Only performs initialization, no additional operations
95+
- **tools_call scenario**: Performs initialization, lists tools, and calls the `add_numbers` tool
96+
97+
This separation ensures that each scenario tests exactly what it's supposed to test without side effects.
98+
99+
### Transport
100+
101+
Uses `HttpClientStreamableHttpTransport` which:
102+
- Implements the latest Streamable HTTP protocol (2025-03-26)
103+
- Uses the standard JDK `HttpClient` (no external HTTP client dependencies)
104+
- Supports protocol version negotiation
105+
- Handles SSE streams for server-to-client notifications
106+
107+
### Client Configuration
108+
109+
The client is configured with:
110+
- Client info: `test-client` version `1.0.0`
111+
- Request timeout: 30 seconds
112+
- Default capabilities (no special features required for basic tests)
113+
114+
### Error Handling
115+
116+
The client:
117+
- Exits with code 0 on success
118+
- Exits with code 1 on failure
119+
- Prints error messages to stderr
120+
- Each scenario handler is independent and self-contained
121+
122+
## Adding New Scenarios
123+
124+
To add support for new scenarios:
125+
126+
1. Add the scenario name to the switch statement in `Main.java`
127+
2. Implement a dedicated handler method (e.g., `runAuthScenario()`, `runElicitationScenario()`)
128+
3. Register the scenario in the available scenarios list in the default case
129+
4. Rebuild the JAR
130+
131+
Example:
132+
```java
133+
case "new-scenario":
134+
runNewScenario(serverUrl);
135+
break;
136+
```
137+
138+
## Next Steps
139+
140+
Future enhancements:
141+
142+
- Add more scenarios (auth, elicitation, etc.)
143+
- Implement a comprehensive "everything-client" pattern
144+
- Add to CI/CD pipeline
145+
- Create expected-failures baseline for known issues

‎conformance-tests/client-jdk-http-client/pom.xml‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,53 @@
2626
<artifactId>mcp</artifactId>
2727
<version>0.18.0-SNAPSHOT</version>
2828
</dependency>
29+
30+
<!-- Logging -->
31+
<dependency>
32+
<groupId>ch.qos.logback</groupId>
33+
<artifactId>logback-classic</artifactId>
34+
<version>${logback.version}</version>
35+
<scope>runtime</scope>
36+
</dependency>
2937
</dependencies>
3038

39+
<build>
40+
<plugins>
41+
<!-- Maven Shade Plugin for creating executable JAR with dependencies -->
42+
<plugin>
43+
<groupId>org.apache.maven.plugins</groupId>
44+
<artifactId>maven-shade-plugin</artifactId>
45+
<version>3.5.1</version>
46+
<executions>
47+
<execution>
48+
<phase>package</phase>
49+
<goals>
50+
<goal>shade</goal>
51+
</goals>
52+
<configuration>
53+
<transformers>
54+
<transformer
55+
implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
56+
<mainClass>io.modelcontextprotocol.conformance.client.Main</mainClass>
57+
</transformer>
58+
<transformer
59+
implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer" />
60+
</transformers>
61+
<filters>
62+
<filter>
63+
<artifact>*:*</artifact>
64+
<excludes>
65+
<exclude>META-INF/*.SF</exclude>
66+
<exclude>META-INF/*.DSA</exclude>
67+
<exclude>META-INF/*.RSA</exclude>
68+
</excludes>
69+
</filter>
70+
</filters>
71+
</configuration>
72+
</execution>
73+
</executions>
74+
</plugin>
75+
</plugins>
76+
</build>
77+
3178
</project>

‎conformance-tests/client-jdk-http-client/src/main/java/io/modelcontextprotocol/conformance/client/Main.java‎

Lines changed: 142 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,150 @@
11
package io.modelcontextprotocol.conformance.client;
22

3+
import java.net.URI;
4+
import java.net.http.HttpClient;
5+
import java.time.Duration;
6+
7+
import io.modelcontextprotocol.client.McpAsyncClient;
8+
import io.modelcontextprotocol.client.McpClient;
9+
import io.modelcontextprotocol.client.transport.HttpClientStreamableHttpTransport;
10+
import io.modelcontextprotocol.spec.McpSchema;
11+
12+
/**
13+
* MCP Conformance Test Client - JDK HTTP Client Implementation
14+
*
15+
* <p>
16+
* This client is designed to work with the MCP conformance test framework. It reads the
17+
* test scenario from the MCP_CONFORMANCE_SCENARIO environment variable and the server URL
18+
* from command-line arguments.
19+
*
20+
* <p>
21+
* Usage: Main &lt;server-url&gt;
22+
*
23+
* @see <a href= "https://github.com/modelcontextprotocol/conformance">MCP Conformance
24+
* Test Framework</a>
25+
*/
326
public class Main {
427

528
public static void main(String[] args) {
6-
System.out.println("MCP Conformance Tests - Client");
29+
if (args.length == 0) {
30+
System.err.println("Usage: Main <server-url>");
31+
System.err.println("The server URL must be provided as the last command-line argument.");
32+
System.err.println("The MCP_CONFORMANCE_SCENARIO environment variable must be set.");
33+
System.exit(1);
34+
}
35+
36+
String scenario = System.getenv("MCP_CONFORMANCE_SCENARIO");
37+
if (scenario == null || scenario.isEmpty()) {
38+
System.err.println("Error: MCP_CONFORMANCE_SCENARIO environment variable is not set");
39+
System.exit(1);
40+
}
41+
42+
String serverUrl = args[args.length - 1];
43+
44+
try {
45+
switch (scenario) {
46+
case "initialize":
47+
runInitializeScenario(serverUrl);
48+
break;
49+
case "tools_call":
50+
runToolsCallScenario(serverUrl);
51+
break;
52+
default:
53+
System.err.println("Unknown scenario: " + scenario);
54+
System.err.println("Available scenarios:");
55+
System.err.println(" - initialize");
56+
System.err.println(" - tools_call");
57+
System.exit(1);
58+
}
59+
System.exit(0);
60+
}
61+
catch (Exception e) {
62+
System.err.println("Error: " + e.getMessage());
63+
e.printStackTrace();
64+
System.exit(1);
65+
}
66+
}
67+
68+
/**
69+
* Helper method to create and configure an MCP client with transport.
70+
* @param serverUrl the URL of the MCP server
71+
* @return configured McpAsyncClient instance
72+
*/
73+
private static McpAsyncClient createClient(String serverUrl) {
74+
HttpClientStreamableHttpTransport transport = HttpClientStreamableHttpTransport.builder(serverUrl).build();
75+
76+
return McpClient.async(transport)
77+
.clientInfo(new McpSchema.Implementation("test-client", "1.0.0"))
78+
.requestTimeout(Duration.ofSeconds(30))
79+
.build();
80+
}
81+
82+
/**
83+
* Initialize scenario: Tests MCP client initialization handshake.
84+
* @param serverUrl the URL of the MCP server
85+
* @throws Exception if any error occurs during execution
86+
*/
87+
private static void runInitializeScenario(String serverUrl) throws Exception {
88+
McpAsyncClient client = createClient(serverUrl);
89+
90+
try {
91+
// Initialize client
92+
client.initialize().block();
93+
94+
System.out.println("Successfully connected to MCP server");
95+
}
96+
finally {
97+
// Close the client (which will close the transport)
98+
client.close();
99+
System.out.println("Connection closed successfully");
100+
}
101+
}
102+
103+
/**
104+
* Tools call scenario: Tests tool listing and invocation functionality.
105+
* @param serverUrl the URL of the MCP server
106+
* @throws Exception if any error occurs during execution
107+
*/
108+
private static void runToolsCallScenario(String serverUrl) throws Exception {
109+
McpAsyncClient client = createClient(serverUrl);
110+
111+
try {
112+
// Initialize client
113+
client.initialize().block();
114+
115+
System.out.println("Successfully connected to MCP server");
116+
117+
// List available tools
118+
McpSchema.ListToolsResult toolsResult = client.listTools().block();
119+
System.out.println("Successfully listed tools");
120+
121+
// Call the add_numbers tool if it exists
122+
if (toolsResult != null && toolsResult.tools() != null) {
123+
for (McpSchema.Tool tool : toolsResult.tools()) {
124+
if ("add_numbers".equals(tool.name())) {
125+
// Call the add_numbers tool with test arguments
126+
var arguments = new java.util.HashMap<String, Object>();
127+
arguments.put("a", 5);
128+
arguments.put("b", 3);
129+
130+
McpSchema.CallToolResult result = client
131+
.callTool(new McpSchema.CallToolRequest("add_numbers", arguments))
132+
.block();
133+
134+
System.out.println("Successfully called add_numbers tool");
135+
if (result != null && result.content() != null) {
136+
System.out.println("Tool result: " + result.content());
137+
}
138+
break;
139+
}
140+
}
141+
}
142+
}
143+
finally {
144+
// Close the client (which will close the transport)
145+
client.close();
146+
System.out.println("Connection closed successfully");
147+
}
7148
}
8149

9150
}

‎conformance-tests/client-jdk-http-client/src/main/resources/logback.xml‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<configuration>
3+
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
4+
<encoder>
5+
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
6+
</encoder>
7+
</appender>
8+
9+
<!-- Set default log level -->
10+
<root level="INFO">
11+
<appender-ref ref="STDOUT" />
12+
</root>
13+
14+
<!-- More verbose logging for MCP client -->
15+
<logger name="io.modelcontextprotocol" level="DEBUG" />
16+
</configuration>

0 commit comments

Comments
 (0)