On Thu, Apr 14, 2016 at 8:05 AM, Andres Freund <andres@anarazel.de> wrote: > > On 2016-04-14 07:59:07 +0530, Amit Kapila wrote: > > What you want to see by prewarming? > > Prewarming appears to greatly reduce the per-run variance on that > machine, making it a lot easier to get meaningful results. >
I think you are referring the tests done by Robert on power-8 m/c, but the performance results I have reported were on intel x86. In last two days, I have spent quite some effort to do the performance testing of this patch with pre-warming by using the same query [1] as used by Robert in his tests. The tests are done such that first it start server, pre-warms the relations, ran read-only test, stop server, again repeat this for next test.
What did you include into single run: test of single version (HEAD or Patch) or test of both of them?
I have observed that the variance in run-to-run performance still occurs especially at higher client count (128). Below are results for 128 client count both when the tests ran first with patch and then with HEAD and vice versa.
Test-1
----------
client count - 128 (basically -c 128 -j 128)
first tests ran with patch and then with HEAD
Patch_ver/Runs
HEAD (commit -70715e6a)
Patch
Run-1
156748
174640
Run-2
151352
150115
Run-3
177940
165269
Test-2
----------
client count - 128 (basically -c 128 -j 128)
first tests ran with HEAD and then with patch
Patch_ver/Runs
HEAD (commit -70715e6a)
Patch
Run-1
173063
151282
Run-2
173187
140676
Run-3
177046
166726
I think this patch (padding pgxact) certainly is beneficial as reported above thread. At very high client count some variation in performance is observed with and without patch, but I feel in general it is a win.
So, what hardware did you use for these tests: power-8 or x86? How long was single run?
Per-run variation seems quite high. It also seems that it depends on which version runs first. But that could be a coincidence.
------ Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company