Обсуждение: PG backup check
Not sure I am following this. Did Google release this because PG backups are not 100% reliable or the data corruption can occur due to hardware failure.
On Mon, Jul 16, 2018 at 8:35 PM, Ravi Krishna <sravikrishna3@gmail.com> wrote:
Not sure I am following this. Did Google release this because PG backups are not 100% reliable or the data corruption can occur due to hardware failure.
The rule of thumb is you should verify that your backups can be restored before you consider them reliable. You don't want to end up in a disaster recovery situation and find that your backups have been corrupted in any way shape or form. Many things can break your backups, especially if they are automated and have many steps involved.
One way to do this is to actually restore the data on a different system and compare. This tool appears to do the comparison for you directly to the dump, increasing confidence that it is reliable.
2018-07-17 2:35 GMT+02:00 Ravi Krishna <sravikrishna3@gmail.com>:
Not sure I am following this. Did Google release this because PG backups are not 100% reliable or the data corruption can occur due to hardware failure.
From what I understand with this Google tool, it has nothing to do with backups. It just allows you to check data blocks in a PostgreSQL cluster. Google advice is to run it before taking a backup, but that's about it.
--
Guillaume.
On Tue, Jul 17, 2018 at 11:17 AM, Guillaume Lelarge <guillaume@lelarge.info> wrote:
2018-07-17 2:35 GMT+02:00 Ravi Krishna <sravikrishna3@gmail.com>:Not sure I am following this. Did Google release this because PG backups are not 100% reliable or the data corruption can occur due to hardware failure.From what I understand with this Google tool, it has nothing to do with backups. It just allows you to check data blocks in a PostgreSQL cluster. Google advice is to run it before taking a backup, but that's about it.
This appears to basically be the same tool that's already included in PostgreSQL 11, and has been around in a few different incarnations (but unpolished) for years.
FWIW, in relation to backups, tools like pgbackrest already did this transparently during backup, and again PostgreSQL 11 will do it built-in.
It's quite possible Google was running this internally before of course, and a separate tool from others, but it's not exactly news... But they do outline a very definite problem, which is that if you get physical corruption in your database, it gets included in the backups. And if it's in a portion of the database you don't use a lot, checksum failures won't be noticed until you actually try, which is way too late.