On Tue, Apr 22, 2008 at 10:31:53AM +0200, Zdenek Kotala wrote:
> When you are able detect ordering difference you are able also check if it
> is important for the test or not without any extra effort. Only what we
> need is put some flag to test that order is not important.
Not true. Sorting the file is going jumble all the results together.
Since we perform many tests in one file, you're not going to be able to
seperate them.
> Regression test MUST BE bulletproof. If you get a error you must know that
> it is really error (in postgresql or regtest) and must be fixed. When you
> start to ignore some errors because it can happen sometimes you fall in the
> trap soon.
I think people are misunderstanding. You posted a bunch of diffs with
that comment that they *appeared* to only be ordering differences. How
good did you check? If an 8 become a 9 chances are you'd miss it.
Having a second test checking the sorted results would at least
preclude the chance that there really is something wrong.
It was a guide, not a way of getting out of tests.
Have a nice day,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/
> Please line up in a tree and maintain the heap invariant while
> boarding. Thank you for flying nlogn airlines.