Guys,
There are two mutually exclusive problems to solve with a
performance-test farm.
The first problem is plaform performance, which would be a matter of
expanding the buildfarm to include a small set of performance tests ...
probably ones based on previously known problems, plus some other simple
common operations. The goal here would be to test on as many different
machines as possible, rather than getting full coverage of peformance.
The second would be to test the full range of PostgreSQL performance.
That is, to test every different thing we can reasonably benchmark on a
PostgreSQL server. This would have to be done on a few dedicated
full-time testing machines, because of the need to use the full hardware
resources. When done, this test would be like a full-blown TPC benchmark.
The above are fundamentally different tests.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com