Hi,
Right now we run tap tests separately in each directory. Which is one of the
reasons the make output is so unreadable - instead of having one 'prove'
output listing all tests, we get a lot of different prove outputs, all
interspersed. And it severely limits parallelism on windows right now.
It's currently not possible to just run all tap tests in one prove run,
because a number of tests assume that they are run from specific directories
and/or with per-directory parameters.
For meson I "solved" this by running each individual test in a wrapper that
changes directory etc. But that's not really a great approach.
To me it seems we should change our tests and test invocations to not depend
on being run from a specific directory and to unify the environment variables
passed to tap tests to one set somewhere central.
I think this would require:
1) Moving handling of PG_TEST_EXTRA into the tap tests themselves. That's a
good idea imo, because there's then output explaining that some tests
aren't run.
2) teach tap test infrastructure to add the directory containing the test to
the perl search path, so that modules like RewindTest.pm can be found.
3) teach tap test infrastructure to compute the output directory for a
specific test from the file location of the test itself and a parameter
like top_builddir.
4) Pass env variables like LZ4, TAR, GZIP_PROGRAM by a mechanism other than
exports in individual makefiles. Maybe just generating a perl file at
configure / mkvcbuild.pl / meson setup time?
While far from a small amount of work, it does seem doable? A good number of
tap tests already pass this way, btw, just not all.
Greetings,
Andres Freund