Re: performance-test farm

Dne 12.5.2011 08:54, Greg Smith napsal(a):
> Tomas Vondra wrote:
>> Actually I was not aware of how the buildfarm works, all I
>> knew was there's something like that because some of the hackers mention
>> a failed build on the mailing list occasionally.
>>
>> So I guess this is a good opportunity to investigate it a bit ;-)
>>
>> Anyway I'm not sure this would give us the kind of environment we need
>> to do benchmarks ... but it's worth to think of.
>>   
> 
> The idea is that buildfarm systems that are known to have a) reasonable
> hardware and b) no other concurrent work going on could also do
> performance tests.  The main benefit of this approach is it avoids
> duplicating all of the system management and source code building work
> needed for any sort of thing like this; just leverage the buildfarm
> parts when they solve similar enough problems.  Someone has actually
> done all that already; source code was last sync'd to the build farm
> master at the end of March:  https://github.com/greg2ndQuadrant/client-code
> 
> By far the #1 thing needed to move this forward from where it's stuck at
> now is someone willing to dig into the web application side of this. 
> We're collecting useful data.  It needs to now be uploaded to the
> server, saved, and then reports of what happened generated.  Eventually
> graphs of performance results over time will be straighforward to
> generate.  But the whole idea requires someone else (not Andrew, who has
> enough to do) sits down and figures out how to extend the web UI with
> these new elements.

Hi all,

it seems CSPUG will get two refurbished servers at the end of this
month. We plan to put both of them to the buildfarm - one for regular
testing with Czech locales and I'd like to use the other one for the
proposed performance testing.

I'm willing to put some time into this, but I'll need help with
preparing the 'action plan' (because you know - I live in EU, and in EU
everything is driven by action plans).

AFAIK what needs to be done is:

1) preparing the hw, OS etc. - ok
2) registering the machine as a buildfarm member - ok
3) modifying the buildfarm client-code to collect performance data
 - What data should to be collected prior to the benchmark?
    a) info about the environment (to make sure it's safe)?    b) something else?
 - What performance tests should be executed?
    a) let's start with pgbench - select-only and regular    b) something else in the future? DSS/DWH workloads?    c)
specialtests (spinlocks, db that fits to RAM, ...)
 

4) modifying the buildfarm server-code to accept and display  performance data
  - not really sure what needs to be done here

regards
Tomas