Re: Eliminating PD_ALL_VISIBLE, take 2

On 2013-07-02 10:12:31 -0700, Jeff Davis wrote:
> On Tue, 2013-07-02 at 14:02 +0200, Andres Freund wrote:
> > Ok, so you want some benchmark results. I spent 20 minutes concocting some
> > quick tests. Here you go:
> > 
> > master (384f933046dc9e9a2b416f5f7b3be30b93587c63):
> > tps = 155075.448341 (including connections establishing)
> > tps = 155259.752267 (excluding connections establishing)
> > 
> > dev (384f933046dc9e9a2b416f5f7b3be30b93587c63 + patch):
> > tps = 151450.387021 (including connections establishing)
> > tps = 152512.741161 (excluding connections establishing)
> > 
> > That's about a 3% regression.
> 
> I had a little trouble reproducing this result on my workstation, and my
> previous tests on the 64-core box didn't seem to show a difference
> (although I didn't spend a lot of time on it, so perhaps I could try
> again).

> I did see some kind of difference, I think. But the fastest run without
> the patch beat the slowest run with the patch by about 1.4%. The delta
> generally seemed closer to 0.5%. The noise seemed to be around 2.6%.

It was more reliably reproduceable here, I ran every test 5 times and
the differences between runs weren't big. But I wouldn't be surprised if
it's dependent on the exact CPU.

> Why did you do this as a concurrent test? The difference between reading
> hints and PD_ALL_VISIBLE doesn't seem to have much to do with
> concurrency.

Primarily because I didn't spend too much time on it and just wanted to
get a feeling for things. I originally wanted to check how much the
additional pin/buffer reference on a small table (i.e. ~33+ pages) is
noticeable, but noticed this before.
Also, cache issues are often easier to notice in concurrent scenarios
where the CPUs are overcommitted since increased cache usage shows up
more prominently and intelligence on the cpu level can save less.

> Regardless, this is at least a concrete issue that I can focus on, and I
> appreciate that. Are scans of small tables the primary objection to this
> patch, or are there others? If I solve it, will this patch make real
> progress?

Well, I still have my doubts that it's a good idea to remove this
knowledge from the page level. And that's not primarily related to
performance. Unfortunately a good part of judging architectural
questions is gut feeling...
The only real argument for the removal I see - removal of one cycle of
dirtying - wouldn't really weigh much if we would combine it with
freezing (which we can do if Robert's forensic freezing patch makes it
in).

Greetings,

Andres Freund

-- Andres Freund                       http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services