Re: tests and meson - test names and file locations

On 12.08.22 18:29, Andres Freund wrote:
>> I don't really understand which problem this solves and how.  Sure, the test
>> output is somewhat complex, but I know where it is and I've never found
>> myself wishing it to be somewhere else.
> I'd like the buildfarm and CI a) use parallelism to run tests (that's why the
> BF is slow) b) show the logfiles for exactly the failed test ([1]). We can of
> course iterate through the whole directory tree, somehow identify which log
> files are for which test, and then select the log files for the failed
> tests. But that's much easier to do then when you have a uniform directory
> hierarchy, where you can test which tests have failed based on the filesystem
> alone.

My initial experiences with testing under meson is that it's quite 
fragile and confusing (unlike the building, which is quite robust and 
understandable).  Some of that is the fault of meson, some of that is 
our implementation.  Surely this can be improved over time, but my 
experience has been that it's not there yet.

The idea that we are going to move all the test output files somewhere 
else at the same time is not appealing to me.  The combination of 
fragile plus can't find the diagnostics is not a good one.

Now, this is my experience; others might have different ones.

Also, is there anything in these proposed changes that couldn't also be 
applied to the old build system?  We are going to be running them in 
parallel for some time.  It would be good if one doesn't have to learn 
two entirely different sets of testing interfaces.