Simon Riggs wrote:
>
> We obviously need a performance build farm and I think everyone accepts
> that. We just need to do it, so that's a given and is something I hope
> to be involved in.
>
>
>
It's on my list ... Had I but world enough and time ...
Performance testing can be bolted onto the exiting buildfarm as an
option. However, performance test machines have some requirements that
pure functional/build test machines don't have: especially stability. A
standard buildfarm client can be put on almost any machine and run
happily. My main workstation runs four buildfarm members including three
in a VM, and I never notice any impact. But a performance test machine
probably needs to be dedicated to just that function. And at least some
members of the performance test machines would need to be higher end
machines. The number of people who can afford such resources is much
lower than those who can run a relatively low impact simple buildfarm
member.
Maybe we also need to talk about running clients elsewhere for
performance testing too.
We also need to talk about what would be a good set of tests to run.
One useful thing this would buy us is a time series of test results so
we could easily see sudden degradations in performance. It must have
been annoying trying to triangulate performance dropoff recently.
cheers
andrew