On Thu, 15 Nov 2001, Peter Eisentraut wrote:
> Bill Studenmund writes:
>
> > I think the general point is that it'd be nice if regression tests didn't
> > report failure due to numerical noise. :-)
>
> Actually, I'm not convinced all of these are strictly numerical noise.
> Differences in the 5th out of 10 decimal places or positive vs. negative
> zero look more like "incorrect optimization" or "incomplete floating point
> implementation". It could be interesting to do these calculations
> symbolically and run the final result to plenty of decimal places to see
> who's right.
I think it would depend on where the result came from. I think the
problems the regression tests hit is that we subtract two nearly-equal
numbers. Like ones which were the same for say 7 of 15 digits. So the
answer we get is only significant to 8 digits, but the display code wants
to print 15. So we get seven digits of questionable lineage. I'm not sure
what the standards say should go into them.
Doing the calculation to "plenty" of decimal places would be really
interesting if we could then map the answer back to the number of bits in
the mantisa in the original number. But I'm not sure how to do that easily
in the regression test.
Take care,
Bill