On Sun, Jul 21, 2024 at 5:04 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> Peter Geoghegan <pg@bowt.ie> writes:
> > On Sun, Jul 21, 2024 at 12:51 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >> I do not think the answer to this is to nag the respective animal
> >> owners to raise PG_TEST_TIMEOUT_DEFAULT. IMV this test is simply
> >> not worth the cycles it takes, at least not for these machines.
>
> > Can't we just move it to PG_TEST_EXTRA? Alongside the existing
> > "xid_wraparound" test?
>
> Perhaps. xid_wraparound seems entirely too slow for what it's
> testing as well, if you ask me, and there's a concurrent thread
> about that test causing problems too.
>
> > There will always be a small number of extremely slow buildfarm
> > animals. Optimizing for things like Raspberry pi animals with SD cards
> > just doesn't seem like a good use of developer time. I really care
> > about keeping the tests fast, but only on platforms that hackers
> > actually use for their development work.
>
> I find this argument completely disingenuous. If a test is slow
> enough to cause timeout failures on slower machines, then it's also
> eating a disproportionate number of cycles in every other check-world
> run --- many of which have humans waiting for them to finish. Caring
> about the runtime of test cases is good for future-you not just
> obsolete buildfarm animals.
>
> I note also that the PG_TEST_EXTRA approach has caused xid_wraparound
> to get next-to-zero buildfarm coverage. If that test is actually
> capable of revealing problems, we're unlikely to find out under the
> status quo.
What is the argument for PG_TEST_EXTRA if it is not running on almost
any buildfarm animals? Are some of those tests valuable for other
reasons than being consistently automatically run (e.g. developer
understanding of how a particular part of code works)?
If they aren't being run, how do we know that they still work (as test
infrastructure changes)? The recovery conflict test is skipped on 15,
which means that backported perl test changes may break it without us
knowing. I don't know if you mean that PG_TEST_EXTRA tests are never
run or just seldom run. If they are never run on the buildfarm, then
they could end up silently breaking too.
- Melanie