On Fri, Oct 17, 2014 at 3:34 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Andres Freund <andres@2ndquadrant.com> writes: > On 2014-10-16 10:06:59 -0400, Tom Lane wrote: >> No, it wasn't. I'm not convinced either that that patch will get in at >> all, or that it has to have regression tests of that particular form, >> or that such a switch would be sufficient to make such tests platform >> independent.
> Why should the EXPLAIN ANALYZE output without timing information be less > consistent for sensibly selected cases than EXPLAIN itself?
To take just one example, the performance numbers that get printed for a sort, such as memory consumption, are undoubtedly platform-dependent. Maybe your idea of "sensibly selected cases" excludes sorting, but I don't find such an argument terribly convincing. I think if we go down this road, we are going to end up with an EXPLAIN that has one hundred parameters turning on and off tiny pieces of the output, none of which are of any great use for anything except the regression tests. I don't want to go there. It would be a lot better to expend the effort on a better regression testing infrastructure that wouldn't *need* bitwise-identical output across platforms. (mysql is ahead of us in that department: they have some hacks for selective matching of the output.)
I saw this, and I was about to ask the same question as Andres.... I think I now see what you're worried about. Next we'd need a flag to disable external disk sort sizes too...
Perhaps we could introduce some sort of wildcard matching in the regression tests. So that we could stick something like:
Execution time: * ms
Into the expected results, though, probably we'd need to come up with some wildcard character which is a bit less common than *
It might be hard to generate a useful diff with this for when a test fails, but maybe it'd be good enough to just run the 2 files through this wildcard matching programme and then just do a diff if it fails.