Re: dynamic shared memory

On Wed, Sep 4, 2013 at 6:38 PM, Jim Nasby <jim@nasby.net> wrote:
>> No, I meant something that would live as long as the postmaster and
>> die when it dies.
>
> ISTM that at some point we'll want to look at putting top-level shared
> memory into this system (ie: allowing dynamic resizing of GUCs that affect
> shared memory size).

A lot of people want that, but being able to resize the shared memory
chunk itself is only the beginning of the problem.  So I wouldn't hold
my breath.

> Wouldn't it protect against a crash while writing the file? I realize the
> odds of that are pretty remote, but AFAIK it wouldn't cost that much to
> write a new file and do an atomic mv...

If there's an OS-level crash, we don't need the state file; the shared
memory will be gone anyway.  And if it's a PostgreSQL-level failure,
this game neither helps nor hurts.

>> Sure.  A messed-up backend can clobber the control segment just as it
>> can clobber anything else in shared memory.  There's really no way
>> around that problem.  If the control segment has been overwritten by a
>> memory stomp, we can't use it to clean up.  There's no way around that
>> problem except to not the control segment, which wouldn't be better.
>
> Are we trying to protect against "memory stomps" when we restart after a
> backend dies? I thought we were just trying to ensure that all shared data
> structures were correct and consistent. If that's the case, then I was
> thinking that by using a pointer that can be updated in a CPU-atomic fashion
> we know we'd never end up with a corrupted entry that was in use; the
> partial write would be to a slot with nothing pointing at it so it could be
> safely reused.

When we restart after a backend dies, shared memory contents are
completely reset, from scratch.  This is true of both the fixed size
shared memory segment and of the dynamic shared memory control
segment.  The only difference is that, with the dynamic shared memory
control segment, we need to use the segment for cleanup before
throwing it out and starting over.  Extra caution is required because
we're examining memory that could hypothetically have been stomped on;
we must not let the postmaster do anything suicidal.

>>> Should dsm_impl_op sanity check the arguments after op? I didn't notice
>>> checks in the type-specific code but I also didn't read all of it... are
>>> we
>>> just depending on the OS to sanity-check?
>>
>> Sanity-check for what?
>
> Presumably there's limits to what the arguments can be rationally set to.
> IIRC there's nothing down-stream that's checking them in our code, so I'm
> guessing we're just depending on the kernel to sanity-check.

Pretty much.  It's possible more thought is needed there, but the
shape of those additional thoughts is not clear to me at this time.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company