HTTPCLIENT-1822: async transparent content decompression #681

arturobernalg · 2025-07-18T13:06:47Z

This patch fulfils HTTPCLIENT-1822 by delivering transparent (de)compression support in the async pipeline.

DecompressingStringAsyncEntityConsumer – decodes gzip/deflate/brotli/zstd … responses using the codecs in ContentCodecRegistry and returns the plain body as a String.

CompressingAsyncEntityProducer – wraps any outgoing producer and re-encodes its payload with the chosen Content-Encoding, also via ContentCodecRegistry; enables gzip (or other) uploads.

AsyncClientDecompressionExample shows response-side usage. AsyncClientCompressionExample demonstrates gzip uploads.

With these additions an async client can both receive and send compressed entities without extra boiler-plate.

...src/main/java/org/apache/hc/client5/http/async/methods/DecompressingAsyncEntityConsumer.java

ok2c · 2025-07-21T10:37:21Z

@arturobernalg I am not sure about this one. I would prefer a proper solution (even if it is GZIP only) without full content buffering or nothing at all.

arturobernalg · 2025-07-24T06:17:19Z

without full content buffering or nothing at all

@ok2c I've implemented streaming DEFLATE and GZIP compression/decompression without full content buffering, . I'm not sure if this fully aligns with what you requested.

garydgregory · 2025-07-24T07:17:57Z

...5/src/main/java/org/apache/hc/client5/http/async/methods/CompressingAsyncEntityProducer.java

+        if (coding == null) {
+            throw new IOException("Unknown coding: " + token);
+        }
+        final java.util.function.UnaryOperator<HttpEntity> op = ContentCodecRegistry.encoder(coding);


This shouldn't need to be a fully qualified name.

garydgregory · 2025-07-24T07:23:16Z

...ava/org/apache/hc/client5/http/async/methods/GzipDecompressingStringAsyncEntityConsumer.java

+    }
+
+    private GzipState state = GzipState.HEADER_MAGIC1;
+    private int flags = 0;


I don't think we should initialize instance variables to default values (like in C!), it feels like clutter.

ok2c · 2025-07-24T10:54:58Z

@arturobernalg I am not in a position to tell you what to do. At the moment I see only DeflateCompressingAsyncEntityProducer as promising.

Why does DeflateDecompressingStringAsyncEntityConsumer work with Strings only? Could not it work with any arbitrary EntityConsumer?

I would focus on these two classes for now and drop the rest.

arturobernalg · 2025-07-24T14:08:04Z

@arturobernalg I am not in a position to tell you what to do. At the moment I see only DeflateCompressingAsyncEntityProducer as promising.

Why does DeflateDecompressingStringAsyncEntityConsumer work with Strings only? Could not it work with any arbitrary EntityConsumer?

I would focus on these two classes for now and drop the rest.

@ok2c please do another pass. I'll implement the gzip in another PR>

ok2c · 2025-07-25T08:14:28Z

@arturobernalg I guess I failed to properly express myself. I actually thought you were going to build DeflatingAsyncEntityProducer and InflatingAsyncEntityCunsumer using Deflater / Inflater API directly without any of InputStream / OutputStream code that basically does not work well with async transport. I can still see HttpEntity / InputStream / OutputStream all over the place. I cannot stop you but i do not find such implementations very useful.

arturobernalg · 2025-07-25T09:58:01Z

build DeflatingAsyncEntityProducer and InflatingAsyncEntityCunsumer using Deflater / Inflater API directly

@ok2c implemented.

ok2c

@arturobernalg DeflatingAsyncEntityProducer and InflatingAsyncEntityCunsumer look good.

I would drop CompressingAsyncEntityProducer.

Could you also add an async version of ContentCompressionExec to the request execution pipeline?

ok2c

@arturobernalg Very good otherwise.

.../src/test/java/org/apache/hc/client5/http/examples/AsyncClientDeflateCompressionExample.java

...client5/src/test/java/org/apache/hc/client5/http/examples/AsyncClientDeflatePostExample.java