On Mon, Jun 13, 2022 at 12:59 PM Aleksander Alekseev
<aleksander@timescale.com> wrote:
> So, to clarify, what we are trying to achieve here is to reduce the
> probability of an event when a page gets corrupted but the checksum is
> accidentally the same as it was before the corruption, correct? And we
> also assume that neither file system nor hardware catched this
> corruption.
Yeah, I think so, although it also depends on what the filesystem and
the hardware would do if they did catch the corruption. If they would
have made our read() or write() operation fail, then any checksum
feature at the PostgreSQL level is superfluous. If they would have
noticed the operation but not caused a failure, and say just logged
something in the system log, a PostgreSQL check could still be useful,
because the PostgreSQL user might not be looking at the system log,
but will definitely notice if they get an ERROR rather than a query
result from PostgreSQL. And if the lower-level systems wouldn't have
caught the failure at all, then checksums are useful in that case,
too.
> If that's the case I would say that using something like SHA256 would
> be an overkill, not only because of the consumed disk space but also
> because SHA256 is expensive. Allowing the user to choose from 16-bit,
> 32-bit and maybe 64-bit checksums should be enough. I would also
> suggest that no matter how we do it, if the user chooses 16-bit
> checksums the performance and the disk consumption should remain as
> they currently are.
If the user wants 16-bit checksums, the feature we've already got
seems good enough -- and, as you say, it doesn't use any extra disk
space. This proposal is just about making people happy if they want a
bigger checksum.
On the topic of which algorithm to use, I'd be inclined to think that
it is going to be more useful to offer checksums that are 64 bits or
more, since IMHO 32 is not all that much more than 16, and I still
think there are going to be alignment issues. Beyond that I don't have
anything against your specific suggestions, but I'd like to hear what
other people think.
--
Robert Haas
EDB: http://www.enterprisedb.com