I'm mostly done with my review of the "Automatic adjustment of
bgwriter_lru_maxpages" patch. In addition to issues already brought up
with that code, there are some small things that need to be done to merge
it with the recent pg_stat_bgwriter patch, and I have some concerns about
its unbounded scanning of the buffer pool; I'll write that up in more
detail or just submit an improved patch as I get time this week.
But there's a fundamental question that has been bugging me, and I think
it impacts the direction that code should take. Unless I'm missing
something in my reading, buffers written out by the LRU writer aren't ever
put onto the free list. I assume this is to stop from prematurely
removing buffers that contain useful data. In cases where a substantial
percentage of the buffer cache is dirty, the LRU writer has to scan a
significant portion of the pool looking for one of the rare clean buffers,
then write it out. When a client goes to grab a free buffer afterward, it
has to scan the same section of the pool to find the now clean buffer,
which seems redundant.
With the new patch, the LRU writer is fairly well bounded in that it
doesn't write out more than it thinks it will need; you shouldn't get into
a situation where many more pages are written than will be used in the
near future. Given that mindset, shouldn't pages the LRU scan writes just
get moved onto the free list?
--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD