Re: Incorrect rounding of double values at max precision

Поиск
Список
Период
Сортировка
От Andres Freund
Тема Re: Incorrect rounding of double values at max precision
Дата
Msg-id 20201021020525.uw6oijt55uymsk5p@alap3.anarazel.de
обсуждение исходный текст
Ответ на Re: Incorrect rounding of double values at max precision  (Tom Lane <tgl@sss.pgh.pa.us>)
Ответы Re: Incorrect rounding of double values at max precision  (Tom Lane <tgl@sss.pgh.pa.us>)
Список pgsql-bugs
Hi,

On 2020-10-20 21:48:52 -0400, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > It seems crazy that we throw away integer precision in the range it's
> > guaranteed to be accurate (that's what, −16777216 to 16777216 for
> > float4, more than a magnitude larger than this value). It does feel
> > different to me that we're throwing away precision that we could know is
> > not just the result of floating point imprecision.
> 
> Meh.  Yeah, we could improve the observed results for float4 values
> that are integers between 1M and 16M, and some similarly-sized band
> for float8; but to what end?

I'm not actually arguing that we should improve it by relying on range
based heuristics. Just that throwing away precision that can't just
argued to have been conjured by float representation issues is an
indicator of our current approach to be quite wrong.


> The most likely practical result is just to postpone the user's
> discovery that they're Doing It Wrong.  If you expect exact answers
> out of float calculations then you are going to learn an expensive
> lesson sooner or later.  Better sooner, before you've stored even more
> inexact data that you cannot fix.

I don't buy this, not even for a second. Why is FLT_DIG = 6 the right
way to hint at that? Why not a precision of 5, 4, 3? Sure 6 digits is
guaranteed to roundtrip the same way, but that's an *extremely* coarse
filter.

And if we like exposing people to floating point imprecision, why is
select (502.15::float8 / 10)::numeric resulting in 50.214999999999996
bad?

Greetings,

Andres Freund



В списке pgsql-bugs по дате отправления:

Предыдущее
От: Tom Lane
Дата:
Сообщение: Re: Incorrect rounding of double values at max precision
Следующее
От: Tom Lane
Дата:
Сообщение: Re: Incorrect rounding of double values at max precision