On Mon, Jan 22, 2018 at 6:53 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Here's a possibly more useful graph of regression test timings over
> the last year. I pulled this from the buildfarm database: it is the
> reported runtime for the "installcheck-C" step in each successful
> build of HEAD on dromedary, going back to Jan. 2017. I picked dromedary
> because I know that that machine hasn't gotten any software updates
> nor is there anything else very interesting going on on it. I dropped
> three or four obvious outlier reports (possibly those ran during the
> machine's nightly backup cron job). The reported runtime is only
> precise to 1s, and a couple seconds jitter is hardly surprising, so
> there's a good deal of noise. Still, it's possible to discern when
> I put some effort into test runtime reduction back in April, and
> it can be seen that things have gotten notably slower since the
> beginning of November.
Right, but this doesn't seem to show any big spike in the runtime at
the time when parallel hash was committed, or when the preparatory
patch to add test coverage for hash joins got committed. Rather,
there's a gradual increase over time. Either we're making the server
slower (which would be bad) or we're adding proper test coverage for
all the new features that we're adding (which would be good). We
can't expect every feature patch to preserve the runtime of the tests
absolutely unchanged; figuring out what can be optimized is a separate
exercise from adding test coverage either for new things or for things
that weren't previously covered.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company