Re: BUG - broken "make check" if different options

>> As I was investing playing around with blocksize, I noticed that some test
>> cases under "make check" vary depending on compilation parameters, as
>> they:
>
> There has never been any expectation that the regression tests would 
> pass exactly no matter what the environment.  If we tried to make them 
> do so, we'd end up restricting the scope of testing so much as to be 
> nearly useless.

Hmmm... ok, so this is a feature.

I would have thought that tests about functional results should always 
pass, no matter what the environment, provided the environment allows to 
run the test, so basicaly queries should return deterministic results, 
implying an ORDER BY when there is more than one row in the output. So for 
me this part looks more like a bug than a feature.

For tests about plans, this is less obvious. Maybe test settings should 
control the environment enough so as to warrant deterministic results 
(say, tell the planner to assume this and this and this, and what is your 
plan ?). That would also help to test plan decisions with more extreme 
hardware. However, this would probably imply a test infrastructure beyond 
what is currently available. So I would be more ok for that part as a 
feature.


Also, this means that changing the default block size is basically never 
tested, otherwise the buildfarm would be reder than it is.

-- 
Fabien.