Re: Same double precision operations, different results
От | Tom Lane |
---|---|
Тема | Re: Same double precision operations, different results |
Дата | |
Msg-id | 6262.1392312408@sss.pgh.pa.us обсуждение исходный текст |
Ответ на | Same double precision operations, different results (Daniel Vázquez <daniel2d2art@gmail.com>) |
Ответы |
Re: Same double precision operations, different results
|
Список | pgsql-hackers |
Daniel Vázquez <daniel2d2art@gmail.com> writes: > Please can someone tell me why this behavior? Thx!! You're confusing numeric and double precision; in particular this calculation is *not* being done in float8, but numeric: > # select (0.766238989559398 * 0.766238989559398 * 1 + 0.642555686986733 * > 0.642555686986733) calc; > (*) Why this calculation produces 1 and not 0.999999999999999633651488135693 Actually, it's not producing 1, but a smidgen more: regression=# set extra_float_digits TO 3; SET regression=# select ( cast ( cos(radians(39.9826557)) * cos(radians(39.9826557)) * cos(radians(-0.04773120000004383) -radians(-0.04773120000004383)) + sin(radians(39.9826557)) * sin(radians(39.9826557)) as double precision )); float8 ---------------------1.00000000000000022 (1 row) You've got roundoff error either way, but this way happens to be in the direction that makes acos() complain. regards, tom lane
В списке pgsql-hackers по дате отправления: