On Sun, Jul 21, 2024 at 5:04 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > There will always be a small number of extremely slow buildfarm
> > animals. Optimizing for things like Raspberry pi animals with SD cards
> > just doesn't seem like a good use of developer time. I really care
> > about keeping the tests fast, but only on platforms that hackers
> > actually use for their development work.
>
> I find this argument completely disingenuous.
Disingenuous? Really?
> If a test is slow
> enough to cause timeout failures on slower machines, then it's also
> eating a disproportionate number of cycles in every other check-world
> run --- many of which have humans waiting for them to finish. Caring
> about the runtime of test cases is good for future-you not just
> obsolete buildfarm animals.
That's not necessarily true, though.
I actually benchmarked this new test. I found that its runtime was a
little on the long side as these recovery TAP tests go, but not to an
excessive degree. It wasn't the slowest by any means.
It's entirely possible that the new test would in fact be far slower
than other comparable tests, were I to run a similar benchmark on
something like a Raspberry pi with an SD card -- that would explain
the apparent inconsistency here. Obviously Raspberry pi type hardware
is expected to be much slower than the machine I use day to day, but
that isn't the only thing that matters. A Raspberry pi can also have
completely different performance characteristics to high quality
workstation hardware. The CPU might be tolerably fast, while I/O is a
huge bottleneck.
> I note also that the PG_TEST_EXTRA approach has caused xid_wraparound
> to get next-to-zero buildfarm coverage. If that test is actually
> capable of revealing problems, we're unlikely to find out under the
> status quo.
I saw that.
I think that there is significant value in providing a way for
individual developers to test wraparound. Both by providing a TAP
test, and providing the associated SQL callable C test functions.
There is less value in testing it on every conceivable platform.
--
Peter Geoghegan