Re: Enabling Checksums

On 04.03.2013 22:40, Jeff Davis wrote:
> Is there any reason why we can't have both postgres and filesystem
> checksums?

Of course not. But if we can get away without checksums in Postgres, 
that's better, because then we don't need to maintain that feature in 
Postgres. If the patch gets committed, it's not mission accomplished. 
There will be discussion and need for further development on things like 
what to do if you get a checksum failure, patches to extend the 
checksums to cover things like the clog and other non-data files and so 
forth. And it's an extra complication that will need to be taken into 
account when developing other new features; in particular, hint bit 
updates need to write a WAL record. Even if you have all the current 
hint bits covered, it's an extra hurdle for future patches that might 
want to have hint bits in e.g new index access methods.

> The same user might not want both (or might, if neither are
> entirely trustworthy yet), but I think it's too early to declare one as
> the "right" solution and the other not. Even with btrfs stable, I
> pointed out a number of reasons users might not want it, and reasons
> that the project should not depend on it.

The PostgreSQL project would not be depending on it, any more than the 
project depends on filesystem snapshots for backup purposes, or the OS 
memory manager for caching.

> Numbers are always nice, but it takes a lot of effort to come up with
> them. What kind of numbers are you looking for, and how *specifically*
> will those numbers affect the decision?

Benchmark of vanilla PostgreSQL, PostgreSQL + this patch, and PostgreSQL 
running on btrfs or ZFS with data checksums enabled. DBT-2 might be a 
good candidate, as it's I/O heavy. That would be a good general test; in 
addition it would be good to see a benchmark of the worst case scenario 
for the fragmentation you're expecting to see on btrfs, as well as a 
worst case scenario for the extra WAL traffic with the patch.

> If btrfs with checksums is 10% slower than ext4 with postgres checksums,
> does that mean we should commit the postgres checksums?

In my opinion, a 10% gain would not be worth it, and we should not 
commit in that case.

> On the other side of the coin, if btrfs with checksums is exactly the
> same speed as ext4 with no postgres checksums (i.e. checksums are free
> if we use btrfs), does that mean postgres checksums should be rejected?

Yes, I think so. I'm sure at least some others will disagree; Greg 
already made it quite clear that he doesn't care how the performance of 
this compares with btrfs.

- Heikki