On 2022-01-23 21:25:04 -0500, Tom Lane wrote:
> Michael Paquier <michael@paquier.xyz> writes:
> > On Sun, Jan 23, 2022 at 06:10:07PM -0800, Noah Misch wrote:
> >> Could do that. Every run that doesn't get the flaky failure will print a
> >> message like "TODO passed: 3-5", though the test file could mitigate that by
> >> declaring the TODO only on configurations where we expect a failure. The
> >> 027_stream_regress.pl trouble involves reaching a die(), not failing a test in
> >> this sense, so that one would take more work.
>
> > Using a TODO has the advantage to allow the tests to run on a periodic
> > basis, even if they could fail in this unexpected way.
>
> I'm okay with this *if* the TODO marking can be constrained to platforms
> where we know there's a problem. Otherwise I'm afraid it will mask
> unexpected problems.
Yep. Very weird syntax... TODO blocks only take effect if a local $TODO is set
(to a text explaining the reason hopefully) or if there's a todo_skip. So
something like
TODO:
{
local $TODO = 'linux + sparc is borked' if $^O eq 'linux' and ...;
tests...
}
would run the tests and accept failures if the if matches, and otherwise not.
Greetings,
Andres Freund