Re: dynamic shared memory

Hi Noah,

On 2013-09-01 09:24:00 -0400, Noah Misch wrote:
> > >> But even if it isn't reliable, there doesn't seem to be all that much
> > >> value in forbidding access to that part of the OS-provided API.  In
>
> That's also valid, though.  Even if no core code exploits the flexibility,
> 3rd-party code might do so.

It seems more likely that 3rd party code misunderstands the
limitations. But perhaps that's being too picky.

> > > I for one cannot see how you even remotely could make that work a) on
> > > windows (check the troubles we have to go through to get s_b
> > > consistently placed, and that's directly after startup) b) 32bit systems.
> >
> > Noah?
>
> The difficulty depends on whether processes other than the segment's creator
> will attach anytime or only as they start.  Attachment at startup is enough
> for parallel query, but it's not enough for something like lock table
> expansion.  I'll focus on the attach-anytime case since it's more general.

Even on startup it might get more complicated than one immediately
imagines on EXEC_BACKEND type platforms because their memory layout
doesn't need to be the same. The more shared memory you need, the harder
that will be. Afair

> On a system supporting MAP_FIXED, implement this by having the postmaster
> reserve address space under a PROT_NONE mapping, then carving out from that
> mapping for each fixed-address dynamic segment.  The size of the reservation
> would be controlled by a GUC; one might set it to several times anticipated
> peak usage.  (The overhead of doing that depends on the kernel.)  Windows
> permits the same technique with its own primitives.

Note that allocating a large mapping, even without using it, has
noticeable cost, at least under linux. The kernel has to create & copy
data to track each pages state (without copying the memory content's
itself due to COW) for every fork afterwards. If you don't believe me,
check the whole discussion about go's (the language) memory
management...

If that's the solution we go for why don't we just always include heaps
more shared memory and use that (remapping part of it as PROT_NONE)
instead of building the infrastructure in this patch?

> A system where mmap() accepts only a zero address in practice (HP-UX,
> according to Gnulib, although HP docs suggest it has improved over time)
> requires a different technique.  For those systems, expand the regular shared
> memory segment and carve from that to make "dynamic" segments.  This amounts
> to adding ShmemFree() to supplement ShmemAlloc().  If a core platform had to
> use this implementation, its disadvantages would be sufficient to discard the
> whole idea of reliable fixed addresses.  But I find it acceptable if it's a
> crutch for older kernels, rare hardware, etc.
>
> I don't foresee fundamental differences on 32-bit.  All the allocation
> maximums scale down, but that's the usual story for 32-bit.

If you actually want to allocate memory after starting up, without
carving a section out for that from the beginning, the memory
fragmentation will make it very hard to find memory addresses of the
same across processes.
If you go for allocating from the start what you end up will not
actually be "dynamic shared memory" because you cannot allocate much
(even if not actually backed by memory!) without compromising
performance. So we will end up with a configurable
"dynamic_shared_memory = ..." parameter. And even then, all processes,
even those not actually using the "dynamic" memory, will have to pay the
price of not being able to allocate as much memory as otherwise possible.

Greetings,

Andres Freund

-- Andres Freund                       http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services