Re: Changing shared_buffers without restart

> On Mon, Jul 14, 2025 at 10:25:51AM +0530, Ashutosh Bapat wrote:
> > Currently it's being addressed via every backend waiting for each other,
> > but I guess it could be as well managed via handling the freelist, so
> > that only "available" buffers will be inserted into the lookup table.
>
> I didn't get how can that be managed by freelist? Buffers are also
> allocated through clocksweep, which needs to be managed as well.

The way it is implemented in the patch right now is new buffers are
added into the freelist right away, when they're initialized by the
virtue of nextFree. What I have in mind is to do this as the last step,
when all backends have confirmed shared memory signal was absorbed. This
would mean that StrategyControll will not return a buffer id from the
freshly allocated range until everything is done, and no such buffer
will be inserted into the buffer lookup table.

You're right of course, a buffer id could be returned from the
ClockSweep and from the custom strategy buffer ring. Buf from what I see
those are picking a buffer from the set of already utilized buffers,
meaning that for a buffer to land there it first has to go through
StrategyControl->firstFreeBuffer, and hence the idea above will be a
requirement for those as well.

> > Long story short, in the next version of the patch I'll try to
> > experiment with a simplified design: a simple function to trigger
> > resizing, launching a coordinator worker, with backends not waiting for
> > each other and buffers first allocated and then marked as "available to
> > use".
>
> Should all the backends wait between buffer allocation and them being
> marked as "available"? I assume that marking them as available means
> "declaring the new NBuffers".

Yep, making buffers available would be equivalent to declaring the new
NBuffers. What I think is important here is to note, that we use two
mechanisms for coordination: the shared structure ShmemControl that
shares the state of operation, and ProcSignal that tells backends to do
something (change the memory mapping). Declaring the new NBuffers could
be done via ShmemControl, atomically applying the new value, instead of
sending a ProcSignal -- this way there is no need for backends to wait,
but StrategyControl would need to use the ShmemControl instead of local
copy of NBuffers. Does it make sense to you?

> What about when shrinking the buffers? Do you plan to make all the
> backends wait while the coordinator is evicting buffers?

No, it was never planned like that, since it could easily end up with
coordinator waiting for the backend to unpin a buffer, and the backend
to wait for a signal from the coordinator.