Add Ser/De + roundtrip protocol benchmarks#6776
Conversation
Benchmarks for all 6 AWS protocol types measuring serialization and deserialization in isolation (no HTTP, signing, or retries): - JSON (DynamoDB PutItem) - REST-JSON (Lambda CreateFunction) - REST-XML (CloudFront CreateDistribution) - Query (STS AssumeRole) - EC2 (EC2 DescribeInstances) - CBOR (CloudWatch GetMetricData) Each protocol has a V1 and V2 benchmark class with @benchmark methods for both ser and deser, using the same JMH configuration and fixture data for fair comparison.
RanVaknin
added
changelog-not-required
RanVaknin
changed the title
|
|
||
| Instant end = Instant.parse("2026-03-09T00:00:00Z"); | ||
| Instant start = end.minusSeconds(3600); | ||
| request = GetMetricDataRequest.builder() |
There was a problem hiding this comment.
In general we consider the request object to be non-reusable in v2 and best practice is for users to create a new request per operation invocation. There is a (small) cost to this that maybe we should be including in these benchmarks. I'm on the fence about it - it's not as apples to apples with v1 benchmarks but it is our best practice guidance to not reuse request objects, so maybe we should capture that cost in the comparison?
There was a problem hiding this comment.
I agree with you on this. Since we are measuring the entire roundtrip it makes sense to include the request building within the benchmark runner. Just pushed the change.
| @Override | ||
| protected void service(HttpServletRequest req, HttpServletResponse resp) throws IOException { | ||
| resp.setStatus(200); | ||
| resp.setContentLength(body.length); |
There was a problem hiding this comment.
We should also probably be setting content-type? I'm not quite sure if it matters or not
|
|
||
| @Benchmark | ||
| public void getMetricDataSer(Blackhole bh) { | ||
| bh.consume(new GetMetricDataRequestProtocolMarshaller(protocolFactory).marshall(request)); |
There was a problem hiding this comment.
I can't remember how the marshallers work here - but do they mutate the request at all? If so, that could leak state between iterations
There was a problem hiding this comment.
As far as I can tell there are no mutations happening. The marshaller just calls .get* on each field and passes it on to the marshalling function. There are no setters that mutate the reqeust (at least in the benchmarks codepath).
I wrote a small test to prove that:
GetMetricDataRequest request = createRequest();
SdkRpcV2CborProtocolFactory protocolFactory = new SdkRpcV2CborProtocolFactory(new RpcV2CborClientMetadata());
String before = request.toString();
new GetMetricDataRequestProtocolMarshaller(protocolFactory).marshall(request);
new GetMetricDataRequestProtocolMarshaller(protocolFactory).marshall(request);
String after = request.toString();
System.out.println(before.equals(after)); // prints true
|
2 participants
Benchmarking notes
For JSON/RestJson/CBOR deserialization, v2's narrowest unmarshalling entry point takes an SdkHttpFullResponse, so the v2 benchmarks create an SdkHttpFullResponse wrapper per iteration. v1's narrowest entry point takes a raw parser, so it doesn't have this overhead. Even though v2's deserialization benchmark is slightly disadvantaged, it is faster across the board.