Re: Improving connection scalability (src/backend/storage/ipc/procarray.c)

On 5/31/22 16:36, Ranier Vilela wrote:
> Em dom., 29 de mai. de 2022 às 17:10, Ranier Vilela <ranier.vf@gmail.com
> <mailto:ranier.vf@gmail.com>> escreveu:
> 
>     Em dom., 29 de mai. de 2022 às 15:21, Andres Freund
>     <andres@anarazel.de <mailto:andres@anarazel.de>> escreveu:
> 
>         On 2022-05-29 18:00:14 +0200, Tomas Vondra wrote:
>         > Also, there's the question of correctness, and I'd bet Andres
>         is right
>         > getting snapshot without holding ProcArrayLock is busted.
> 
>         Unless there's some actual analysis of this by Rainier, I'm just
>         going to
>         ignore this thread going forward. It's pointless to invest time when
>         everything we say is just ignored.
> 
>     Sorry, just not my intention to ignore this important point.
>     Of course, any performance gain is good, but robustness comes first.
> 
>     As soon as I have some time.
> 
> I redid the benchmarks, with getting a snapshot with holding ProcArrayLock.
> 
> Average Results
>     
>     
>     
>     
> Connections:         
>     tps head     tps patch     diff     
> 1     39196,3088985     39858,0207936     661,711895100008     101,69%
> 2     65050,8643819     65245,9852367     195,1208548     100,30%
> 5     91486,0298359     91862,9026528     376,872816899995     100,41%
> 10     131318,0774955     131547,1404573     229,062961799995     100,17%
> 50     116531,2334144     116687,0325522     155,799137800001     100,13%
> 100     98969,4650449     98808,6778717     -160,787173199991     99,84%
> 200     89514,5238649     89463,6196075     -50,904257400005     99,94%
> 300     88426,3612183     88457,2695151     30,9082968000002     100,03%
> 400     88078,1686912     88338,2859163     260,117225099995     100,30%
> 500     87791,1620039     88074,3418504     283,179846500003     100,32%
> 600     87552,3343394     87930,8645184     378,530178999994     100,43%
> 1000     86538,3772895     86771,1946099     232,817320400005     100,27%
> avg     89204,4088731917     89420,444631825     1981,0816042     100,24%
> 
>  
> For clients with 1 connections, the results are good.

Isn't that a bit strange, considering the aim of this patch was
scalability? Which should improve higher client counts in the first place.

> But for clients with 100 and 200 connections, the results are not good.
> I can't say why these two tests were so bad.
> Because, 100 and 200 results, I'm not sure if this should go ahead, if
> it's worth the effort.
> 

I'd argue this is either just noise, and there's no actual difference.
This could be verified by some sort of statistical testing (e.g. the
well known t-test).

Another option is that this is simply due to differences in binary
layout - this can result in small differences (easily 1-2%) that are
completely unrelated to what the patch does. This is exactly what the
"stabilizer" talk I mentioned a couple days ago was about.

FWIW, when a patch improves scalability, the improvement usually
increases with the number of clients. So you'd see no/small improvement
for 10 clients, 100 clients would be improved more, 200 more, etc. We
see nothing like that here. So either the patch does not really improve
anything, or perhaps the benchmark doesn't even hit the bottleneck the
patch is meant to improve (which was already suggested in this thread
repeatedly).


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company