Re: Re: [GENERAL] Trouble with float4 after upgrading from 6.5.3 to 7.0.2

At 12:11 7/08/00 -0400, Tom Lane wrote:
>Philip Warner <pjw@rhyme.com.au> writes:
>> pjw=# select float8(float4(10.1));
>>       float8
>> ------------------
>>  10.1000003814697
>> (1 row)
>
>> I would have expected the latter to be at worst 10.10000000000000 +/-
>> .00000000000001.
>
>float4 is good to about 7 decimal digits (24 mantissa bits) on
>IEEE-standard machines.  Thus the above result is actually closer
>than you have any right to expect.
>
>Don't they teach people about float arithmetic in CS 101 anymore?
>

No idea. It's a couple of decades since I did it.

I wasn't complaining about the float4 accuracy; I was complaining about the
way it was converted to float8. It seems more intuitive to zero-extend base
10 zeros...




----------------------------------------------------------------
Philip Warner                    |     __---_____
Albatross Consulting Pty. Ltd.   |----/       -  \
(A.C.N. 008 659 498)             |          /(@)   ______---_
Tel: (+61) 0500 83 82 81         |                 _________  \
Fax: (+61) 0500 83 82 82         |                 ___________ |
Http://www.rhyme.com.au          |                /           \|
                                 |    --________--
PGP key available upon request,  |  /
and from pgp5.ai.mit.edu:11371   |/