Re: A test for replay of regression tests

On Sat, Jan 15, 2022 at 12:49 PM Andres Freund <andres@anarazel.de> wrote:
> On 2022-01-15 02:32:35 +1300, Thomas Munro wrote:
> > 1. The way I invoke pg_regress doesn't seem to work correctly under
> > the build farm client (though it works fine under make), not sure why
> > yet, but reproduced here and will figure it out tomorrow.
>
> I think it's just a problem of the buildfarm specifying port names in
> extra_opts. E.g.
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=eelpout&dt=2022-01-14%2011%3A49%3A36
> has
>
> # Checking port 58074
> # Found port 58074
> Name: primary
> ...
> # Running: /home/tmunro/build-farm/buildroot/HEAD/pgsql.build/src/test/recovery/../../../src/test/regress/pg_regress
--dlpath="/home/tmunro/build-farm/buildroot/HEAD/pgsql.build/src/test/regress"--bindir= --port=58074
--schedule=../regress/parallel_schedule--max-concurrent-tests=20 --inputdir=../regress
--outputdir="/home/tmunro/build-farm/buildroot/HEAD/pgsql.build/src/test/recovery"--port=5678 
> (using postmaster on /tmp/1W6qVPVyCv, port 5678)
>
> Note how there's both --port=58074 and --port=5678 in the pg_regress
> invocation. The latter comming from EXTRA_REGRESS_OPTS, which the buildfarm
> client sets.
>
> The quickest fix would probably be to just move the 027_stream_regress.pl
> added --port until after $extra_opts?

Thanks, I figured it was an environment variable biting me, and indeed
it was that one.  I reordered the arguments, tested locally under the
buildfarm client script, and pushed.  I'll keep an eye on the build
farm.

One thing I noticed is that the pg_dump output files should really be
rm'd by the clean target; I may push something for that later.

> > 2. The new test in src/test/modules/t/002_tablespace.pl apparently has
> > some path-related problem on Windows
>
> This is the failure you're talking about?
>
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2022-01-14%2012%3A04%3A55
>
> > that I didn't know about, because CI isn't even running the TAP tests under
> > src/test/module/test_misc (and various other locations), but the BF is :-/
> > And I was happy because modulescheck was passing...
>
> This we need to fix... But if you're talking about fairywren's failure, it's
> more than not running some tests, it's that we do not test windows mingw
> outside of cross compilation.

I'm temporarily stumped by complete ignorance of MSYS.  I tried the
test on plain old Windows/MSVC by cherry-picking the reverted commit
d1511fe1 and running .\src\tools\msvc\vcregress.bat taptest
.\src\test\modules\test_misc in my Windows 10 VM, and that passed with
flying colours (so Windows CI would have passed too, if we weren't
ignoring TAP tests in unusual locations, I assume).  I'll look into
installing MSYS to work this out if necessary, but it may take me a
few days.

Here's how it failed on fairywren, in case someone knowledgeable of
MSYS path translation etc can spot the problem:

psql:<stdin>:1: ERROR:  directory
"/home/pgrunner/bf/root/HEAD/pgsql.build/src/test/modules/test_misc/tmp_check/t_002_tablespace_main_data/ts1"
does not exist
not ok 1 - create tablespace with absolute path

I think that means chmod() failed with ENOENT.  That's weird, because
the .pl does:

+my $TS1_LOCATION = $node->basedir() . "/ts1";
+mkdir($TS1_LOCATION);