[discussion from -committers]
On 3/26/20 4:31 PM, Tom Lane wrote:
> Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
>> On 3/26/20 11:31 AM, Tom Lane wrote:
>>> Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
>>>> I don't think this belongs in installcheck, we should add
>>>> 'NO_INSTALLCHECK = 1' to the Makefile.
>>> Why? The other src/test/modules/ modules with TAP tests do not
>>> specify that, with the exception of commit_ts which has a solid
>>> doesnt-work-in-the-default-configuration excuse.
>> That seems wrong, installcheck should be testing against an installed
>> instance, and the TAP tests don't.
> So? We clearly document that for the TAP tests, "make installcheck"
> means "use the installed executables, but run a new instance" [1].
I think we were probably a bit shortsighted about that. But what's done
is done. I wonder if there is a simple way we could turn it off for the
buildfarm?
>> Moreover, from the buildfarm's POV
>> it's completely wrong, as we call the installcheck targets multiple
>> times, once for each configured locale. See one of the animals that
>> tests multiple locales (e.g. crake or prion)
> Yeah. That's productive if you think the tests might be
> locale-sensitive. I doubt that any of the ones under src/test/modules/
> actually are at the moment, so maybe this is a waste of buildfarm effort.
> But I don't think that it's the place of the Makefiles to dictate such
> policy, and especially not for them to do so by breaking the ability to
> use "make installcheck" at all.
>
> regards, tom lane
>
> [1] https://www.postgresql.org/docs/devel/regress-tap.html
Right now the explicit TAP test code in the buildfarm knows how to collect all the relevant output. The installcheck
codedoesn't know about that for TAP tests.
I get that developers want to be able to run tests in a small number of commands, but for the buildfarm I generally
favormore disaggregated tests. That way if test X fails it's much easier to focus on the problem. (related note: I'm
workingon breaking up the log text blobs which will also help focussing on the right area).
Maybe we need to take the discussion to -hackers.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services