Skip to content

Conversation

@rschmitt
Copy link
Contributor

@rschmitt rschmitt commented Jun 5, 2025

The HTTP client builders always construct a client with a non-null cookie spec registry; if a custom instance is not specified, then the builders will use CookieSpecSupport.createDefault(), which loads the full public suffix list. The PSL rules are stored on the heap as a ConcurrentHashMap, i.e. a giant array (16,384 on my machine) containing map entries, each of which is also allocated as a separate object on the heap. This is a considerable number of live objects that the garbage collector may need to trace. I ran a simple microbenchmark to show the effects of loading the PSL on garbage collection (these numbers are from JDK 1.8):

GC took 2 ms
GC took 2 ms
GC took 2 ms
Loading public suffix list... done (took 193 ms)
GC took 11 ms
GC took 8 ms
GC took 7 ms

This is potentially a significant amount of tail latency for use cases that don't require cookie management, such as RPC calls. With this change, it is now simpler to configure and construct a client that does not load the PSL into memory.

The HTTP client builders always construct a client with a non-null
cookie spec registry; if a custom instance is not specified, then the
builders will use `CookieSpecSupport.createDefault()`, which loads the
full public suffix list. The PSL rules are stored on the heap as a
`ConcurrentHashMap`, i.e. a giant array (16,384 on my machine)
containing map entries, each of which is also allocated as a separate
object on the heap. This is a considerable number of live objects that
the garbage collector may need to trace. I ran a simple microbenchmark
to show the effects of loading the PSL on garbage collection (these
numbers are from JDK 1.8):

    GC took 2 ms
    GC took 2 ms
    GC took 2 ms
    Loading public suffix list... done (took 193 ms)
    GC took 11 ms
    GC took 8 ms
    GC took 7 ms

This is potentially a significant amount of tail latency for use cases
that don't require cookie management, such as RPC calls. With this
change, it is now simpler to configure and construct a client that does
not load the PSL into memory.
@arturobernalg
Copy link
Member

@rschmitt — shouldn’t we apply the same change to HttpClientBuilder as well?

@rschmitt
Copy link
Contributor Author

rschmitt commented Jun 5, 2025

@PascalSchumacher
Copy link

👍 but in most cases the full public suffix list will be loaded by DefaultHostnameVerifier later on.

@rschmitt
Copy link
Contributor Author

rschmitt commented Jun 5, 2025

@PascalSchumacher I think this is the wrong default behavior and I plan on dealing with it next.

@rschmitt rschmitt merged commit ac19392 into apache:master Jun 5, 2025
10 checks passed
@rschmitt rschmitt deleted the no-psl branch June 19, 2025 15:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants