Re: 16-bit page checksums for 9.2

On 22.02.2012 14:30, Simon Riggs wrote:
> On Wed, Feb 22, 2012 at 7:06 AM, Noah Misch<noah@leadboat.com>  wrote:
>> On Sun, Feb 19, 2012 at 05:04:06PM -0500, Robert Haas wrote:
>>> Another disadvantage of the current scheme is that there's no
>>> particularly easy way to know that your whole cluster has checksums.
>>> No matter how we implement checksums, you'll have to rewrite every
>>> table in the cluster in order to get them fully turned on.  But with
>>> the current design, there's no easy way to know how much of the
>>> cluster is actually checksummed.  If you shut checksums off, they'll
>>> linger until those pages are rewritten, and there's no easy way to
>>> find the relations from which they need to be removed, either.
>>
>> I'm not seeing value in rewriting pages to remove checksums, as opposed to
>> just ignoring those checksums going forward.  Did you have a particular
>> scenario in mind?
>
> Agreed. No reason to change a checksum unless we rewrite the block, no
> matter whether page_checksums is on or off.

This can happen:

1. checksums are initially enabled. A page is written, with a correct 
checksum.
2. checksums turned off.
3. A hint bit is set on the page.
4. While the page is being written out, someone pulls the power cord, 
and you get a torn write. The hint bit change made it to disk, but the 
clearing of the checksum in the page header did not.
5. Sometime after restart, checksums are turned back on.

The page now has an incorrect checksum on it. The next time it's read, 
you get a checksum error.

I'm pretty uncomfortable with this idea of having a flag on the page 
itself to indicate whether it has a checksum or not. No matter how many 
bits we use for that flag. You can never be quite sure that all your 
data is covered by the checksum, and there's a lot of room for subtle 
bugs like the above, where a page is reported as corrupt when it isn't, 
or vice versa.

This thing needs to be reliable and robust. The purpose of a checksum is 
to have an extra sanity check, to detect faulty hardware. If it's 
complicated, whenever you get a checksum mismatch, you'll be wondering 
if you have broken hardware or if you just bumped on a PostgreSQL bug. I 
think you need a flag in pg_control or somewhere to indicate whether 
checksums are currently enabled or disabled, and a mechanism to scan and 
rewrite all the pages with checksums, before they are verified.

I've said this before, but I still don't like the hacks with the version 
number in the page header. Even if it works, I would much prefer the 
straightforward option of extending the page header for the new field. 
Yes, it means you have to deal with pg_upgrade, but it's a hurdle we'll 
have to jump at some point anyway.

--   Heikki Linnakangas  EnterpriseDB   http://www.enterprisedb.com