TDigest quantile estimate algorithm correctness

[Jefffrey]

BTW, the estimate quantile algorithm doesn't follow the paper, any reason for this?

https://github.com/apache/arrow-datafusion/blob/df8aa7a2e2a6f54acfbfed336b84144256fb7ff8/datafusion/physical-expr/src/aggregate/tdigest.rs#L523-L524

Originally posted by @HaoYang670 in #4259

[Jefffrey]

More recent code link:

datafusion/datafusion/functions-aggregate-common/src/tdigest.rs

Lines 531 to 532 in 8c8e565

	let value = self.centroids[pos].mean()
	+ ((rank - t) / self.centroids[pos].weight() - 0.5) * delta;

Some context:

It was part of the original code which in turn was ported from another repo, see:

https://github.com/MnO2/t-digest/blob/2d27fc5ae977afd95c012b080540091349151ca3/src/lib.rs#L503

May need @HaoYang670 to add more details here, creating issue so we don't lose track of the comment

[LiaCastaneda]

Hey we observed some strannge results using approx_precentile (we saw results where 90>p99), this fix was suggested #21007 NVM me, it's a wrong repro 🤦‍♀️ I'll close the PR