[RFC, POC] Don't require a NBuffer sized PrivateRefCount array of local buffer pins

Hi,

I've been annoyed at the amount of memory used by the backend local
PrivateRefCount array for a couple of reasons:

a) The performance impact of AtEOXact_Buffers() on Assert() enabled
   builds is really, really annoying.
b) On larger nodes, the L1/2/3 cache impact of randomly accessing
   several megabyte big array at a high frequency is noticeable. I've
   seen the access to that to be the primary (yes, really) source of
   pipeline stalls.
c) On nodes with significant shared_memory the sum of the per-backend
   arrays is a significant amount of memory, that could very well be
   used more beneficially.

So what I have done in the attached proof of concept is to have a small
(8 currently) array of (buffer, pincount) that's searched linearly when
the refcount of a buffer is needed. When more than 8 buffers are pinned
a hashtable is used to lookup the values.

That seems to work fairly well. On the few tests I could run on my
laptop - I've done this during a flight - it's a small performance win
in all cases I could test. While saving a fair amount of memory.

Alternatively we could just get rid of the idea of tracking this per
backend, relying on tracking via resource managers...

Greetings,

Andres Freund

--
 Andres Freund                       http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services