On 2024-Jul-22, Melanie Plageman wrote:
> On Sun, Jul 21, 2024 at 5:04 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > I note also that the PG_TEST_EXTRA approach has caused xid_wraparound
> > to get next-to-zero buildfarm coverage. If that test is actually
> > capable of revealing problems, we're unlikely to find out under the
> > status quo.
>
> What is the argument for PG_TEST_EXTRA if it is not running on almost
> any buildfarm animals? Are some of those tests valuable for other
> reasons than being consistently automatically run (e.g. developer
> understanding of how a particular part of code works)?
I think it's a bad idea to require buildfarm owners to edit their config
files as we add tests that depend on PG_TEST_EXTRA. AFAIR we invented
that setting so that tests that had security implications could be made
opt-in instead of opt-out; I think this was a sensible thing to do, to
avoid possibly compromising the machines in some way. But I think these
new tests have a different problem, so we shouldn't use the same
mechanism.
What about some brainstorming to improve this?
For example: have something in the tree that lets committers opt some
tests out from specific BF machines without having to poke at the BF
machines. I imagine two files: one that carries tags for buildfarm
members, something like the /etc/groups file,
src/test/tags.lst
slow: gull,mamba,mereswine,copperhead
and another file that lists tests to skip on members that have certain
tags,
src/tools/buildfarm/do_not_run.lst
slow:src/test/modules/xid_wraparound
slow:src/test/recovery/t/043_vacuum_horizon_floor.pl
so that run_build.pl know that if the current member has tag slow, then
these two tests are to be skipped.
Then we can have xid_wraparound enabled generally (without requiring
PG_TEST_EXTRA), and the BF client knows not to run it in the particular
cases where it's not wanted.
This proposal has a number of problems (a glaring one being the
maintenance of the list of members per tag), but maybe it inspires
better ideas.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"Siempre hay que alimentar a los dioses, aunque la tierra esté seca" (Orual)