Re: pg_shmem_allocations view

On 2014-05-05 15:04:07 -0400, Robert Haas wrote:
> On Sun, May 4, 2014 at 7:50 AM, Andres Freund <andres@2ndquadrant.com> wrote:
> > On 2014-05-04 13:44:17 +0200, Andres Freund wrote:
> >> postgres=# SELECT * FROM pg_shmem_allocations ORDER BY size DESC;
> >>                  key                 |     off     |    size     | allocated
> >> -------------------------------------+-------------+-------------+-----------
> >>  Buffer Blocks                       |   286242528 | 17179869184 | t
> >>  Buffer Descriptors                  |   152024800 |   134217728 | t
> >> ...
> >>  OldSerXidControlData                | 17584357344 |          16 | t
> >> (44 rows)
> >
> > Thinking about this, I think it was a mistake to not add a 'name' field
> > to dynamic shared memory's dsm_control_item. Right now it's very hard to
> > figure out which extension allocated a dsm segment. Imo we should change
> > that before 9.4 is out. I am not suggesting to use it to identify
> > segments, but just as an identifier, passed in into dsm_create().
> >
> > Imo there should be a corresponding pg_dynshmem_allocations to
> > pg_shmem_allocations.
> 
> Well, right now a dsm_control_item is 8 bytes.  If we add a name field
> of our usual 64 bytes, they'll each be 9 times bigger.  We're not
> talking about a lot of bytes in absolute terms, but I guess I'm not in
> favor of an 800% size increase without somewhat more justification
> than you've provided here.  Who is using dynamic shared memory for
> enough different things at this point to get confused?

The kernel side overhead of creating a shared memory segment are so much
higher that this really isn't a meaningful saving. Also, are you really
considering a couple hundred bytes to be a problem?
I think it's quite a sensible thing for an administrator to ask where
all the memory has gone. The more users for dsm there the more important
that'll get. Right now pretty much the only thing a admin could do is to
poke around in /proc to see which backend has mapped the segment and try
to figure out via the logs what caused it to do so. Not nice.

> I'm quite in favor of having something like this for the main shared
> memory segment, but I think that's 9.5 material at this point.

Clearly. For one the version I posted here missed allocations which
aren't done via ShmemInitStruct (lwlock main array and hash table
allocations primarily). For another it's too late ;)

Greetings,

Andres Freund

-- Andres Freund                       http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services