Re: How much do the hint bits help?

On Wed, 2010-12-22 at 16:22 +0200, Heikki Linnakangas wrote:
> On 22.12.2010 15:59, Simon Riggs wrote:
> > On Wed, 2010-12-22 at 15:30 +0200, Heikki Linnakangas wrote:
> >> My gut feeling is that a reasonable compromise is to set hint bits like
> >> we do today, but don't mark the page as dirty when only hint bits are
> >> set. That way you get the benefit of hint bits for tuples that are
> >> frequently accessed and stay in buffer cache. But you don't spend any
> >> extra I/O to set them. I'd really like to see a worst-case scenario
> >> benchmark of a patch that does that.
> >
> > That sounds great, but still prevents block checksums and that is a very
> > valuable feature for robustness.
> 
> It does? The problem with block checksums is that if you modify a page 
> and don't have a corresponding WAL record for it, like a hint bit 
> update, you can have a torn page so that the checksum doesn't match. 
> Refraining from dirtying the page when a hint bit is updated avoids the 
> problem. With that change, we only ever write pages to disk that have a 
> WAL record associated with it, with full-page images as necessary to 
> avoid torn pages.

Which then leads to a block CRC not matching the block in memory. Sure,
we can avoid CRC checking the hint bits, but that requires a much more
expensive and complex CRC check.

So what you suggest works only if we restrict CRC checking to blocks
incoming to the buffer cache, but leaves us unable to do CRC checks on
blocks once in the buffer cache. Since many blocks stay in cache almost
constantly, we're left with the situation that the most heavily used
parts of the database seldom get CRC checked.

Postgres needs CRC checking more than it needs hint bits.

I think we should allow this as an option, and if it proves to be an
issue during beta then we can remove it before we go live, assuming we
cannot get a reasonable alternate optimisation.

I think its important for Postgres to implement this in the same release
as sync rep. They complement each other: confirmed robustness. Exactly
the features we need to prove to the rest of the world to trust us with
their data.

-- Simon Riggs           http://www.2ndQuadrant.com/books/PostgreSQL Development, 24x7 Support, Training and Services