Re: calculating an aspect of shared buffer state from a background worker

Поиск
Список
Период
Сортировка
От Robert Berry
Тема Re: calculating an aspect of shared buffer state from a background worker
Дата
Msg-id CAM=RetHbzKpTa9ad0N6Pq42xnrb_FxP2oF7trCgx1nCO01dmXw@mail.gmail.com
обсуждение исходный текст
Ответ на Re: calculating an aspect of shared buffer state from a background worker  (Tom Lane <tgl@sss.pgh.pa.us>)
Список pgsql-hackers
Thank you both for the thoughtful and helpful responses.

The utility of the length of the free list is somewhat dubious.  I imagine it could be useful to answer the question of "is there a chance that increasing shared buffers would be useless?" in an optimization context.  Agreed it's not useful in most steady state scenarios.

I saw the approach in the pg_buffercache contrib module and am looking for lockless alternatives for at least estimating the size of free buffers.

I'm a relatively inexperienced so I'd be curious to know whether there is a danger beyond an inconsistent result in traversing / sampling the BufferDescriptors without a lock?

Also I got the impression that there is a ring approach to freeing buffers, and that assuming the descriptors are allocated in sequential addresses, taking the difference in the first and last could be used to get a rough estimate accounting for sizes or other shenanigans?  

Thank you again, the clues to look at buffer descriptors and ShmemInitStruct are very helpful.

Best Regards,
Robert


On Mon, Mar 10, 2014 at 7:33 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Robert Berry <berrydigital@gmail.com> writes:
> I'm looking at doing a calculation to determine the number of free buffers
> available.  A n example ratio that is based on some data structures in
> freelist.c as follows:

> (StrategyControl->lastFreeBuffer - StrategyControl->firstFreeBuffer) /
> (double) NBuffers

> Is there a way to get access to the StrategyControl pointer in the context
> of a background worker?

The BufferStrategyControl struct is in shared memory, so you can certainly
get at it.  One way would be to modify freelist.c to export its static
pointer variable.  Alternatively, you could call ShmemInitStruct an extra
time to look up the struct for yourself, and then save it in your own
static variable.

Having said that, though, I'm pretty dubious of the premise.  I trust you
realize that the above calculation is entirely wrong; firstFreeBuffer and
lastFreeBuffer are list head and tail pointers, and have no numerical
relation to the list length.  The only way to determine the list length
accurately would be to chase down the whole list, which you'd have to hold
the BufFreelistLock while doing, which'd be disastrous for performance if
the list was long.  (If you're okay with modifying the backend code you
could dodge this by teaching freelist.c to maintain a counter, I guess.)

An even bigger issue is that it's not clear that the length of the free
list is actually a useful number to have; in steady-state usage it
frequently is always zero.  Buffers only get put back on the freelist if
they're invalidated, eg by dropping the relation they belonged to.  Normal
usage tends to allocate buffers by reclaiming ones whose usage_count has
reached zero in the clock sweep algorithm.  So a better picture of the
availability of buffers would require scanning the buffer pool to see how
many there are of each usage_count level.

                        regards, tom lane

В списке pgsql-hackers по дате отправления:

Предыдущее
От: Alvaro Herrera
Дата:
Сообщение: Re: on_exit_reset fails to clear DSM-related exit actions
Следующее
От: Josh Berkus
Дата:
Сообщение: Re: Changeset Extraction v7.9.1