Re: Performance degradation in commit 6150a1b0

On Wed, Apr 13, 2016 at 9:10 AM, Robert Haas <robertmhaas@gmail.com> wrote:
>
> On Tue, Apr 12, 2016 at 10:30 PM, Noah Misch <noah@leadboat.com> wrote:
> > That sounds like this open item is ready for CLOSE_WAIT status; is it?
>
> I just retested this on power2.  Here are the results.  I retested
> 3fed4174 and 6150a1b0 plus master as of deb71fa9.  5-minute pgbench -S
> runs, scale factor 300, with predictable prewarming to minimize
> variation, as well as numactl --interleave.  Each result is a median
> of three.
>
> 1 client: 3fed4174 = 13701.014931, 6150a1b0 = 13669.626916, master =
> 19685.571089
> 8 clients: 3fed4174 = 126676.357079, 6150a1b0 = 125239.911105, master
> = 122940.079404
> 32 clients: 3fed4174 = 323989.685428, 6150a1b0 = 338638.095126, master
> = 333656.861590
> 64 clients: 3fed4174 = 495434.372578, 6150a1b0 = 457794.475129, master
> = 493034.922791
> 128 clients: 3fed4174 = 376412.090366, 6150a1b0 = 363157.294391,
> master = 625498.280370
>
> On this test 8, 32, and 64 clients are coming out about the same as
> 3fed4174, but 1 client and 128 clients are dramatically improved with
> current master.  The 1-client result is a lot more surprising than the
> 128-client result; I don't know what's going on there.  But anyway I
> don't see a regression here.
>
> So, yes, I would say this should go to CLOSE_WAIT at this point,
> unless Amit or somebody else turns up further evidence of a continuing
> issue here.
>

Yes, I also think that this particular issue can be closed.  However I felt that the observation related to performance variation is still present as I never need to perform prewarm or anything else to get consistent results during my work in 9.5 or early 9.6.  Also, Andres, Alexander and myself are working on similar observation (run-to-run performance variation) in a nearby thread [1].