Re: less expensive pg_buffercache on big shmem

Hi Ivan,

This patch needs a rebase, as 06d7fd6e bumped the version to 1.2.

On 09/02/2016 01:38 PM, Ivan Kartyshov wrote:
> On 09/02/2016 06:01 AM, Robert Haas wrote:
>> I wonder whether we ought to just switch from the consistent method to
>> the semiconsistent method and call it good.  I agree with you that
>> taking every buffer partition lock simultaneously seems like too much
>> locking.  And in the future if we replace the buffer mapping hash with
>> something that is lock-free or close to it, then we wouldn't even have
>> buffer partition locks any more, and probably no way at all to get an
>> entirely consistent snapshot.
>> What do you think of this?
>
> I fully agree with you that it would be preferred in the future to
> replace the buffer mapping hash with some of lock-free algorithms.
>
> In the question of replacing the consistent method I agree with you,
> Andres Freund and Peter Geoghegan: the consistent method does not bring
> any considerable benefits.
>
> You might be right regarding the three different modes, but our DBAs
> asked if we could preserve a legacy mode too, thus the choice.
>
> On 09/02/2016 06:19 AM, Andres Freund wrote:
>  > +1. I think, before long, we're going to have to switch away from having
>  > locks & partitions in the first place. So I don't see a problem relaxing
>  > this. It's not like that consistency really buys you anything...  I'd
>  > even consider not using any locks.
>
> If we will replace consistent method, then we should replace it with the
> partially consistent method (called "nonconsistent") because:
> 1) it's based on fast spinlocks (it's not fully up to its name, though)

In other words, it does exactly the thing proposed up-thread, i.e. 
locking only buffer headers.

What do you mean by fast spinlocks? And why aren't they up to the name?

> 2) it's *almost* the fastest one (the less time needed for execution of
> method, the less data core will change and as a consequence the more
> consistent snapshot will be)

I'm not sure I understand what you're saying here? What do you mean by 
"almost the fastest one"?

I'm a bit confused by the results you reported before, i.e. that

1) nonconsistent is 10% faster then consistent method
2) semiconsistent is 5-times slower then nonconsistent method

What does that mean? Are you refering to duration of the queries reading 
data from pg_buffercache, or to impact on the other queries?

How can be semiconsistent 5x slower than nonconsistent? Wouldn't that 
make it significantly slower than the consistent method?

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services