Re: PG_TEST_EXTRA and meson

On Wed, Jul 17, 2024 at 3:31 AM Nazir Bilal Yavuz <byavuz81@gmail.com> wrote:
>
> Hi,
>
> On Wed, 17 Jul 2024 at 00:27, Jacob Champion
> <jacob.champion@enterprisedb.com> wrote:
> >
> > On Tue, Jul 16, 2024 at 2:12 PM Nazir Bilal Yavuz <byavuz81@gmail.com> wrote:
> > >
> > > 2- If PG_TEST_EXTRA is set from the setup command, use it from the
> > > setup command and discard the environment variable. If PG_TEST_EXTRA
> > > is not set from the setup command, then use it from the environment.
> >
> > Is there a way for the environment to override the Meson setting
> > rather than vice-versa? My vote would be to have both available, but
> > with the implementation in patch 2 I'd still have to reconfigure if I
> > wanted to change my test setup.
>
> I think something like attached does the trick. I did not test it
> extensively but it passed the couple of tests I tried.

Thanks a lot for working on this.

I tested this patch with xid_wraparound. It seems to be working.
$ mts xid_wraparound
... snip
1/6 postgresql:setup / tmp_install
    OK                0.36s
2/6 postgresql:setup / install_test_files
    OK                0.10s
3/6 postgresql:setup / initdb_cache
    OK                1.14s
4/6 postgresql:xid_wraparound / xid_wraparound/001_emergency_vacuum
    SKIP              0.14s
5/6 postgresql:xid_wraparound / xid_wraparound/003_wraparounds
    SKIP              0.14s
6/6 postgresql:xid_wraparound / xid_wraparound/002_limits
    SKIP              0.14s

... snip

$ PG_TEST_EXTRA=xid_wraparound mts xid_wraparound
... snip
1/6 postgresql:setup / tmp_install
    OK                0.38s
2/6 postgresql:setup / install_test_files
    OK                0.07s
3/6 postgresql:setup / initdb_cache
    OK                1.13s
4/6 postgresql:xid_wraparound / xid_wraparound/001_emergency_vacuum
    OK               67.33s   7 subtests passed
5/6 postgresql:xid_wraparound / xid_wraparound/002_limits
    OK               70.14s   3 subtests passed
6/6 postgresql:xid_wraparound / xid_wraparound/003_wraparounds
    OK              178.01s   1 subtests passed

... snip

$ mts xid_wraparound
... snip
1/6 postgresql:setup / tmp_install
    OK                0.36s
2/6 postgresql:setup / install_test_files
    OK                0.06s
3/6 postgresql:setup / initdb_cache
    OK                1.14s
4/6 postgresql:xid_wraparound / xid_wraparound/001_emergency_vacuum
    SKIP              0.18s
5/6 postgresql:xid_wraparound / xid_wraparound/002_limits
    SKIP              0.19s
6/6 postgresql:xid_wraparound / xid_wraparound/003_wraparounds
    SKIP              0.19s

... snip

Providing PG_TEST_EXTRA as a configuration option works as well. But I
find it confusing
$ mts xid_wraparound
... snip
1/6 postgresql:setup / tmp_install
    OK                0.71s
2/6 postgresql:setup / install_test_files
    OK                0.06s
3/6 postgresql:setup / initdb_cache
    OK                1.08s
4/6 postgresql:xid_wraparound / xid_wraparound/001_emergency_vacuum
    OK               52.73s   7 subtests passed
5/6 postgresql:xid_wraparound / xid_wraparound/002_limits
    OK               56.36s   3 subtests passed
6/6 postgresql:xid_wraparound / xid_wraparound/003_wraparounds
    OK              160.46s   1 subtests passed
... snip

$ PG_TEST_EXTRA=ldap mts xid_wraparound
... snip
1/6 postgresql:setup / tmp_install
    OK                0.37s
2/6 postgresql:setup / install_test_files
    OK                0.08s
3/6 postgresql:setup / initdb_cache
    OK                1.16s
4/6 postgresql:xid_wraparound / xid_wraparound/001_emergency_vacuum
    SKIP              0.14s
5/6 postgresql:xid_wraparound / xid_wraparound/003_wraparounds
    SKIP              0.15s
6/6 postgresql:xid_wraparound / xid_wraparound/002_limits
    SKIP              0.15s
... snip
$ PG_TEST_EXTRA=xid_wraparound mts xid_wraparound
... snip
1/6 postgresql:setup / tmp_install
    OK                0.36s
2/6 postgresql:setup / install_test_files
    OK                0.06s
3/6 postgresql:setup / initdb_cache
    OK                1.12s
4/6 postgresql:xid_wraparound / xid_wraparound/001_emergency_vacuum
    OK               62.53s   7 subtests passed
5/6 postgresql:xid_wraparound / xid_wraparound/002_limits
    OK               69.78s   3 subtests passed
6/6 postgresql:xid_wraparound / xid_wraparound/003_wraparounds
    OK              186.78s   1 subtests passed

xid_wraparound tests are run if PG_TEST_EXTRA contains xid_wraparound
or it is not set. Any other setting will not run xid_wraparound test.
That's how the patch is coded but it isn't intuitive that changing
whether a test is run by default would require configuring the build
again. Probably we should just get rid of config time PG_TEST_EXTRA
altogether.

I am including +Tristan Partin who knows meson better.

If you are willing to work on this further, please add it to the commitfest.

--
Best Wishes,
Ashutosh Bapat