Hi!
Thanks, Nikolay, I'm pleased to read good things about project I maintain.
I'll add some Pros and Cons of WAL-G from my side.
Pros:
WAL-G is very efficient. At Y.Cloud we have few petabytes of PG, 12K+ of hosts. Each cluster is backuped every night
(withdeltas) and network cost us real moneys (our datacenters are in different countries BTW). This is our data and we
highlyvalue each byte. So we aim to make WAL-G 100% safe and tested.
WAL-G works with PG, MySQL, MongoDB, FDB and MS SQL Server. And almost any storage (cloud, file system or scp). WAL-G
doesnot need any extra space near database.
Cons:
We are not PG vendor. We contribute to PG ecosystem to make our systems better and reduce risks for Yandex.Cloud
customers.You are not expected to buy commercial support for WAL-G from us directly. Though community behind WAL-G is
bigenough and there are many hackers who can implement feature you want.
And as a result, docs are not very detailed. We would appreciate any docs enhancements.
> 8 февр. 2021 г., в 10:23, Victor Sudakov <vas@sibptus.ru> написал(а):
>
> Nikolay Samokhvalov wrote:
>> In short, it's reliable and battle-tested. It's used in companies such as
>> Yandex.Cloud and GitLab.com, successfully.
>
> Nikolay, don't you mind another question?
>
> I'm used to the fact that "pg_basebackup -X" creates a self-sufficient
> backup of a cluster which can be started right away as it contains all
> the WAL files required for recovery. `touch recovery.signal` is never necessary,
> and `touch standby.signal` is optional (when you do PITR etc).
>
> It's not the case with wal-g, the result of the `wal-g backup-fetch`
> command requires `touch recovery.signal` and a restore_command
> configured to fetch WALs from the wal-g storage.
>
> I have also noticed that wal-g keeps pg_control in a separate tar
> archive, and keeps a lot of metadata.
pg_control is in separate file to prevent incomplete restoration. If for some reason WAL-G was killed during
restorationyou won't be able to start resulting corrupted DB.
> The questions are:
>
> 1. If the metadata in the wal-g storage ever becomes corrupt, will I be
> able to restore the database manually from the archives in
> $WALG_*_PREFIX/{basebackups,wal}_005/ ?
Yes, technically you can restore full backup of PG database without metadata at all. You will need to create empty
sentineljson file though.
>
> 2. Is there a `wal-g backup-fetch` option for truly self-sufficient
> restoration?
We would appreciate a pull request for this feature. All you need is to read LSN of start and stop, and run wal-fetch
forevery segment in between from WAL-G process.
Thanks!
Best regards, Andrey Borodin.