On Tue, 2007-11-27 at 13:32 -0500, Andrew Dunstan wrote:
> We also need to talk about what would be a good set of tests to run.
I think we should develop a series of performance regression tests that
can be run as an option on the buildfarm. We'd want a separate page for
that with graphs etc, as you suggest.
My vision for that is a set of tests that test very specific aspects of
code, much the same way as the regression tests attempt feature
coverage. Examples would be
- 10000 INSERTs
- 10000 INSERTs using multi-VALUEs clauses
- 100000 rows inserted by COPY
- 100000 rows inserted by CTAS
We would need a way to compare results between releases, so we can see
which aspects have regressed/improved, just as we have with the
buildfarm. That will also be food for release notes, where we can
mention all actions that are >5% faster, or anything we must regrettably
report as being slower.
Sounds like it's waiting on somebody to make the first move, so maybe I
should do that, then let everybody else chip into the framework.
Should we do this as part of core, or as a separate pgfoundry project?
-- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com