Noah Misch <noah@leadboat.com> writes:
> On Mon, Jan 24, 2022 at 12:49:16PM +1300, Thomas Munro wrote:
>> Trying out a new idea: what if we could tell the buildfarm website
>> that a certain test is currently expected to fail for reasons we can't
>> fix yet (configuration change needed but owner not responding, or
>> bugfix from another project needed, etc)? That could cause it to be
>> displayed in a different shade of green, or grey, or whatever? Other
>> kinds of failures would still show as red. Perhaps this would be
>> configured with a file in a git repo that any committer can push to.
> That would be a better capability to use if we had it, agreed. Is it feasible
> to acquire that capability soon enough?
It's not merely a website issue: you'd really rather that the
buildfarm animal runs the rest of the tests rather than going belly-up
after an expected failure. I think your suggestion about skipping
problematic tests based on an environment variable is more practical
in the near term. We already have some cases like that, too, eg in
src/bin/psql/t/010_tab_completion.pl.
In the long term I could get behind having some less ad-hoc way
of skipping tests, but I don't think we can have that quickly.
regards, tom lane