Re: Page replacement algorithm in buffer cache

On 2013-04-02 12:56:56 -0400, Tom Lane wrote:
> Andres Freund <andres@2ndquadrant.com> writes:
> > On 2013-04-02 12:22:03 -0400, Tom Lane wrote:
> >> I agree in general, though I'm not sure the bgwriter process can
> >> reasonably handle this need along with what it's already supposed to be
> >> doing.  We may need another background process that is just responsible
> >> for keeping the freelist populated.
> 
> > What else is the bgwriter actually doing otherwise? Sure, it doesn't put the
> > pages on the freelist, but otherwise its trying to do the above isn't it?
> 
> I think it will be problematic to tie buffer-undirtying to putting both
> clean and dirty buffers into the freelist.  It might chance to work all
> right to use one scan process for both, but I'm afraid it's more likely
> that we'd end up either overserving one goal or underserving the other.

Hm. I had imagined that we would only ever put clean buffers into the
freelist and that we would never write out a buffer that we don't need
for a new page. I don't see much point in randomly writing out buffers
that won't be needed for something else soon. Currently we can't do much
better than basically undirtying random buffers since we don't really know
which page will reach a usagecount of zero since bgwriter doesn't
manipulate usagecounts.

One other scenario I can see is the problem of strategy buffer reusage
of dirtied pages (hint bits, pruning) during seqscans where we would
benefit from pages being written out fast, but I can't imagine that that
could be handled very well by something like the bgwriter?

Am I missing something?

> Also note the entire design of BgBufferSync right now is predicated on
> the assumption that the rate of motion of the scan strategy point
> reflects the rate at which new buffers are needed.  If backends are
> supposed to always get new buffers from the freelist, that logic becomes
> circular: the bgwriter would be watching itself.  Perhaps we can
> refactor the feedback control loop logic so that the buffer scan rate is
> driven by changes in the length of the freelist, but I'm not sure
> exactly what the consequences would be.

Yea, that will definitely need refactoring. What I am imagining is that
that pacing is basically built ontop of a few StragetyControl variables
like:
* number of buffers from the freelist
* number of buffers acquired by backend because freelist was empty
* number of buffers written out by backend because freelist was empty

Those should be pretty cheap to maintain and should be enough to
implement sensible pacing for bgwriter.

Greetings,

Andres Freund

-- Andres Freund                       http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services