Re: Possible rounding error of large float values?

Поиск
Список
Период
Сортировка
От Oliver Elphick
Тема Re: Possible rounding error of large float values?
Дата
Msg-id 1093378875.15248.39.camel@linda
обсуждение исходный текст
Ответ на Re: Possible rounding error of large float values?  (Josh Berkus <josh@agliodbs.com>)
Ответы Re: Possible rounding error of large float values?  (Jan Wieck <JanWieck@Yahoo.com>)
Список pgsql-sql
On Tue, 2004-08-24 at 20:52, Josh Berkus wrote:
> Adam,
> 
> > 9223372036854775807
> >
> > It gets selected out as:
> >
> > 9.22337203685478E18
> 
> This is a property of FLOAT data types.  They round.
> 
> > Which appears to be rounded. When we cast it to numeric type we get:
> >
> > 9223372036854780000
> >
> > Which also is rounded. It is still possible to find the row using the
> > original value
> 
> Hmmm ... is 15 digits the limit of NUMERIC?   It may be.

It must be the limit of float.  Numeric can't produce any more than it
was given and it was cast from float.

bray=# select 653596708775675750507850507570708696432 ::numeric;                numeric
-----------------------------------------653596708775675750507850507570708696432
(1 row)

-- 
Oliver Elphick                                          olly@lfix.co.uk
Isle of Wight                              http://www.lfix.co.uk/oliver
GPG: 1024D/A54310EA  92C8 39E7 280E 3631 3F0E  1EC0 5664 7A2F A543 10EA
========================================   "I saw in the night visions, and, behold, one like the      Son of man came
withthe clouds of heaven, and came to     the Ancient of days, and they brought him near before      him. And there was
givenhim dominion, and glory, and      a kingdom, that all people, nations, and languages,      should serve him; his
dominionis an everlasting      dominion, which shall not pass away, and his kingdom      that which shall not be
destroyed."                                       Daniel 7:13,14
 



В списке pgsql-sql по дате отправления:

Предыдущее
От: Josh Berkus
Дата:
Сообщение: Re: Possible rounding error of large float values?
Следующее
От: Jan Wieck
Дата:
Сообщение: Re: Possible rounding error of large float values?