HttpsSupport: Omit public suffix matcher from default hostname verifier

[rschmitt]

The use of public suffix matching as part of hostname verification is nonstandard. I can't find anything in the TLS specifications that prescribe or even mention this behavior, having checked:

• RFC 8446: The Transport Layer Security (TLS) Protocol Version 1.3
• RFC 9110: HTTP Semantics
• RFC 9525: Service Identity in TLS

There are of course rules for wildcard matching, but ultimately the question of whether to trust a certificate for *.com is up to the CAs in your trust store. Given the oddity of the PSL matching behavior and the non-trivial runtime overhead of loading and querying the PSL, I think it makes more sense for the default HostnameVerifier to not use this behavior.

[garydgregory]

@rschmitt
I think we need a new test to track regressions.

[ok2c]

non-trivial runtime overhead of loading and querying the PSL

@rschmitt TLS represents non-trivial runtime overhead in general and the hostname verification is a very marginal part of it.

I am not in favor of disabling hostname validation against PLS by default. It is not useless.

What we can do, though, is to switch HostnameVerificationPolicy to BUILTIN by default and disable our own hostname verification code entirely. The users would have to opt in by choosing CLIENT or BOTH or explicitly setting an HttpClientHostnameVerifier at construction time.

[arturobernalg]

**I don’t see any justification for removing public-suffix matching here. Cookie management and TLS hostname verification are orthogonal concerns, and bundling them together is confusing.;

[ok2c]

**I don’t see any justification for removing public-suffix matching here. Cookie management and TLS hostname verification are orthogonal concerns, and bundling them together is confusing.;

@arturobernalg I can probably explain the confusion. The PSL can be used by both the cookie policy and the host name verifier. One may choose to disable PSL checks either for the state management, the TLS security, or both. What @rschmitt is proposing is to disable it for the TLS security by default (not that I agree with the proposal) but there is no risk of Cookie management and TLS hostname verification coupling here.

[rschmitt]

@ok2c I'm not familiar with HostnameVerificationPolicy. It looks like it is used to enable the built-in verifier (via SSLParameters), as well as to choose the default HostnameVerifier instance during building/construction. What are the behavioral differences between CLIENT and BUILTIN? Are there interactions here with other features, like Conscrypt?

[ok2c]

@ok2c I'm not familiar with HostnameVerificationPolicy. It looks like it is used to enable the built-in verifier (via SSLParameters), as well as to choose the default HostnameVerifier instance during building/construction. What are the behavioral differences between CLIENT and BUILTIN? Are there interactions here with other features, like Conscrypt?

@rschmitt Precisely. The CLIENT mode implies the check gets performed by the client (our own hostname verification code) after the TLS handshake completion. The BUILTIN mode implies the check gets performed by the Security Provider as a part of the TLS handshake (which makes more sense). BOTH means both checks get performed.

Using BUILTIN by default would enable to start getting out the whole hostname verification business. At some point we may even deprecate our own code. And there will be no PSL loaded by default.

HostnameVerificationPolicy is generic and is not specific to any Security Providers.

[garydgregory]

Wouldn't updated Javadocs (class or package) help future readers? There is a lot of good information in this thread.

[ok2c]

@rschmitt Apparently the RFC that specifies the expected client behavior is 6125. It does not explicitly mention the PSL but it is not a stretch to suggest that clients are expected to take public suffices into account when checking wildcards in certificates. The best course of action would not have to do the hostname verification at all.

[rschmitt]

Using BUILTIN by default would enable to start getting out the whole hostname verification business. At some point we may even deprecate our own code. And there will be no PSL loaded by default.

This doesn't sound unreasonable, although I do use client-mode verification for some advanced use cases that I wouldn't know how to reimplement using the JDK verifier.

[ok2c]

This doesn't sound unreasonable, although I do use client-mode verification for some advanced use cases that I wouldn't know how to reimplement using the JDK verifier.

@rschmitt There is nothing stopping you from making use of fa custom hostname verifier in addition to the built-in one or instead of it.

[ok2c]

@rschmitt Here's what I propose instead #649

[ok2c]

Superseded by d89fdfe