Peter Geoghegan <pg@heroku.com> writes:
> On Fri, Jul 1, 2016 at 8:37 PM, Noah Misch <noah@leadboat.com> wrote:
>> PostgreSQL is open to moving features from zero test coverage to nonzero test
>> coverage. The last several releases have each done so. Does that
>> sufficiently clarify the policy landscape?
> Not quite, no. There are costs associated with adding regression tests
> that exercise external sorting. What are the costs, and how are they
> felt? How can we add more extensive test coverage without burdening
> those that run extremely slow buildfarm animals, for example?
As the owner of several slow buildfarm critters, and as somebody who runs
the regression tests manually many times on a typical work day, I do have
concerns about that ;-). I agree that improving code coverage of our
tests is a good goal, but I think we need to take steps to make sure that
we don't increase the runtime of the regression tests too much.
I suggest that we should not take the existing code behavior as something
immutable if it makes it hard to create tests. Peter and I already
discussed changing the behavior of hash CREATE INDEX to make its sort code
path more easily reachable, and perhaps there are similar things that
could be done elsewhere. For instance, I do not believe that the minimum
value of maintenance_work_mem was ever graven on a stone tablet; if it
would help to reduce that, let's reduce it.
> I think that a new tuplesort.sql test file is where a test like this
> belongs (not in the existing cluster.sql test file).
If you are thinking of setting it up like numeric_big, I'm not terribly
excited about that, because to the best of my recollection numeric_big
has never identified a bug. It doesn't get run often enough to have
much chance of that.
It's possible that it'd be sensible to have a mechanism like that but
make it opt-out rather than opt-in; that is, the buildfarm critters
would run all tests by default, but owners of slow animals could set
some flag to skip longer tests. Don't know how much new infrastructure
that would take, or whether it's worth the trouble. But I generally
don't believe that long tests are good just for being long. If something
is slow enough that people start taking measures to avoid running it,
that's a net loss not a net win.
regards, tom lane