Re: Floating point error

Поиск
Список
Период
Сортировка
От James Cloos
Тема Re: Floating point error
Дата
Msg-id m3r4k07wu8.fsf@carbon.jhcloos.org
обсуждение исходный текст
Ответ на Floating point error  (Tom Duffey <tduffey@trillitech.com>)
Ответы Re: Floating point error
Список pgsql-general
>>>>> "TD" == Tom Duffey <tduffey@trillitech.com> writes:

TD> Riddle me this. I have a database column of type "real" that gets
TD> mapped to a Java field of type double via JDBC. ...

TD> - Selecting values from both test and production DBs using psql
TD>   shows "10.3885" as the value

TD> - The Java app on production shows "10.3884573" while the test app
TD>   shows "10.3885"

I suspect the issue is that psql(1) and whatever java method you use to
convert the floats to text choose different rounding.

By default, it seems that psql(1) uses something like printf("%.4f",...)
whereas your java app calls a routing which works more like "%.7f".

(The wire format for floats is the same as they are stored, not a text
representation thereof.)

-JimC
--
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

В списке pgsql-general по дате отправления:

Предыдущее
От: Chris Hanks
Дата:
Сообщение: Re: Poor performance when using a window function in a view
Следующее
От: Tom Duffey
Дата:
Сообщение: Re: Floating point error