`NormalDist.overlap()` returns incorrect values when one distribution has an extremely large mean, violating both its doc and mathematical correctness.

[T90REAL]

Bug report

Bug description:

The test is as following:

from statistics import NormalDist

nd1 = NormalDist(1.3e+303, 1.0)  # Extreme but valid mu

nd2_a = NormalDist(0.0, 1.0)
print(f"sigma2=1:   {nd1.overlap(nd2_a)}")   # 0.0 (correct by chance)

nd2_b = NormalDist(0.0, 100.0)
print(f"sigma2=100: {nd1.overlap(nd2_b)}")   # 1.0 (WRONG!)

nd2_c = NormalDist(0.0, 372.0)
print(f"sigma2=372: {nd1.overlap(nd2_c)}")   # nan (violates API)

sigma2=1:   0.0
sigma2=100: 1.0
sigma2=372: nan

It outputs nan.

CPython versions tested on:

3.12

Operating systems tested on:

Linux

[skirpichev]

I originally explored the parameter space to find edge cases.

Is this motivated some practical needs?

The reason is obvious:

cpython/Lib/statistics.py

Lines 1315 to 1318 in 248ce9f

	a = X._mu * Y_var - Y._mu * X_var
	b = X._sigma * Y._sigma * sqrt(dm * dm + dv * log(Y_var / X_var))
	x1 = (a + b) / dv
	x2 = (a - b) / dv

There overflow in both L1315 and L1316, then in L1317 we subtract infinities.

It outputs nan, which has already violated both the doc and definition.

Please note, that CPython docs are here: https://docs.python.org/

The second test returned 1.0, claiming 100% overlap between two distributions whose means are separated by 1.3e+303. I think this is not what is expected.

In second case - only b overflowed.

I think we can fix such corner cases, but at cost of complications, which will slowdown normal code flow. I'm not sure if it worth. CC @rhettinger

Though, maybe this can be reclassified as a documentation issue. Surely, similar affects many functions in the statistics module, when you are using float's.

[T90REAL]

Thank you for the detailed analysis. I found this through testing. It is indeed very unlikely in real-world use. Perhaps it is only necessary to ensure that the actual behavior remains consistent with the doc.

[skirpichev]

On the ground of issues like #132667, I'm thinking about closing this as "not planned".

I also don't see some way to improve docs. Unless something too broad and generic: "floating-point maths used - Here be dragons."

PS: Maybe it's just another case, where floating-point contexts could be useful. With access to the floating-point exceptions flags we could e.g. check in the end if the FE_OVERFLOW was set during computations and then raise OverflowError. Sometimes this could be more appropriate then specific processing for corner cases.

[rhettinger]

I also don't see some way to improve docs. Unless something too broad and generic: "floating-point maths used - Here be dragons."

I concur. Marking as closed.