As a followup to my previous message, here's a response curve on a 48
core server I used at my last job.
https://picasaweb.google.com/lh/photo/aPYHPWPivPsS79fG3AKtZNMTjNZETYmyPJy0liipFm0?feat=directlink
Note the peak at around 38 to 48 cores. This is the sweetspot on this
server for connections. If I allow this server to get to 68 to 70 hard
working connections my throughput drops by half. It's far better to
have inbound connections sit in a queue in a connection pooler and
wait their turn than to have them all clobber this server at one time.
Of course the issue here is active connections, not idle ones. So if
you have say 150 idle and 50 active connections on this server you'd
be doing fine. Until load started to climb. Then as the number of
active connections went past 50, it would get slower in a non-linear
fashion. Since the throughput is half, at 70 or so connections each
query would now be 1/4th or so as fast as they had been when we had 35
or so.
So it's a good idea to get some idea of where that sweet spot is on
your server and stay under it with a good pooler.