Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
> [discussion from -committers]
> On 3/26/20 4:31 PM, Tom Lane wrote:
>> So? We clearly document that for the TAP tests, "make installcheck"
>> means "use the installed executables, but run a new instance" [1].
> I think we were probably a bit shortsighted about that. But what's done
> is done. I wonder if there is a simple way we could turn it off for the
> buildfarm?
I think it was entirely intentional. I use "installcheck" all the time
to save the cost of repeatedly building an install tree. If anything,
it's more important to be able to do that when running a specific
subdirectory's tests than when testing the whole tree, because the
overhead would be worse in proportion. So sprinkling NO_INSTALLCHECK
liberally would make me sad. (In fact, I wonder if we should treat
that as only disabling traditional-framework tests not TAP tests.
The problem of tests requiring atypical configurations doesn't apply
to TAP tests.)
> Right now the explicit TAP test code in the buildfarm knows how to collect all the relevant output. The installcheck
codedoesn't know about that for TAP tests.
It seems like what the buildfarm would like is a way to invoke TAP tests
and traditional-framework tests separately, so that it could apply special
tooling to the former. I'd have no objection to making that possible.
Alternatively, maybe you could just dig through the tree after-the-fact
looking for tmp_check subdirectories, and capturing their contents?
A separate issue is whether or not it's worth running all the
src/test/modules/ tests over again for multiple locales. ISTM the
answer is mostly "no", but I grant that some of them might be locale
sensitive. Maybe we could mark the ones that are in their Makefiles?
Get the buildfarm to look for "LOCALE_SENSITIVE = 1" or the like.
Right now, the modules/ tests don't run long enough for this to be super
important, but we might be more worried about their cost in future.
regards, tom lane