Re: Clock sweep not caching enough B-Tree leaf pages?

On Wed, Apr 16, 2014 at 5:00 AM, Peter Geoghegan <pg@heroku.com> wrote:
> On Tue, Apr 15, 2014 at 3:59 PM, Ants Aasma <ants@cybertec.at> wrote:
>> There's a paper on a non blocking GCLOCK algorithm, that does lock
>> free clock sweep and buffer pinning[7]. If we decide to stay with
>> GCLOCK it may be interesting, although I still believe that some
>> variant of buffer nailing[8] is a better idea, my experience shows
>> that most of the locking overhead is cache line bouncing ignoring the
>> extreme cases where our naive spinlock implementation blows up.
>
> You might be right about that, but lets handle one problem at a time.
> Who knows what the bottleneck will end up being if and when we address
> the naivety around frequency? I want to better characterize that
> problem first.

Just to summarize you about the previous discussion and the
improvements that we decided to do in this area based on feedback
are as follows:

1. Bgwriter needs to be improved so that it can help in reducing   usage count and finding next victim buffer (run the
clocksweep   and add buffers to the free list).
 
2. SetLatch for bgwriter (wakeup bgwriter) when elements in freelist   are less.
3. Split the workdone globallock (Buffreelist) in StrategyGetBuffer   (a spinlock for the freelist, and an lwlock for
theclock sweep).   Here we can try to make it lock free based on atomic ops as   well.
 
4. Bgwriter needs to be more aggressive, logic based on which it   calculates how many buffers it needs to process
needsto be   improved.
 
5. Contention around buffer mapping locks.
6. Cacheline bouncing around the buffer header spinlocks, is there   anything we can do to reduce this?
7. Choose Optimistically used buffer in StrategyGetBuffer().
8. Don't bump the usage count every time buffer is pinned.

I have already addressed some of these improvements in patch[1]
and for other's, I have plan to work on them for 9.5.

I think here you want to address the improvements related to usage
count and see if it can get us win in some of commonly used scenario's,
without affecting any other commonly used scenario.  I feel this is good
idea to pursue and see if we can get good benefits with it.

Infact few days back, I had ran some tests manually to see the
problems around BufFreeListLock (currently I don't have script ready)
and more recently Jason Petersen has done some benchmarking
in this area which you can refer it here[2].

I wonder if we can work together to improve things in this area.

[1]
http://www.postgresql.org/message-id/006e01ce926c$c7768680$56639380$@kapila@huawei.com
[2]
https://googledrive.com/host/0Bx33JCTmOADOeTIwaE9KX21yWEk/Concurrency%20Limits%20with%20Large%20Working%20Sets

With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com