Re: Updated backup APIs for non-exclusive backups

Поиск
Список
Период
Сортировка
От Stephen Frost
Тема Re: Updated backup APIs for non-exclusive backups
Дата
Msg-id CAOuzzgouW-Y8xNaL-Qq8hrME5S96PrKf8LGYVF6xF2BWFUih0Q@mail.gmail.com
обсуждение исходный текст
Ответ на Re: Updated backup APIs for non-exclusive backups  (Laurenz Albe <laurenz.albe@cybertec.at>)
Ответы Re: Updated backup APIs for non-exclusive backups  (Laurenz Albe <laurenz.albe@cybertec.at>)
Список pgsql-hackers
Greetings,

On Sun, Nov 25, 2018 at 15:45 Laurenz Albe <laurenz.albe@cybertec.at> wrote:
Stephen Frost wrote:
> > > On restore, you're
> > > going to need to create a recovery.conf (at least in released versions)
> > > which provides a restore command (needed even in HEAD today) to get the
> > > old WAL, so having to also create the backup_label file shouldn't be
> > > that difficult.
> >
> > You write "recovery.conf" upon recovery, when you have the restored
> > backup, so you have it on a file system.  No problem adding a file then.
> >
> > This is entirely different from adding a "backup_label" file to
> > a backup that has been taken by a backup software in some arbitrary
> > format in some arbitrary location (think snapshot).
>
> There isn’t any need to write the backup label before you restore the database,
> just as you write recovery.conf then.

Granted.
But it is pretty convenient, and writing it to the data directory right away
is a good thing on top, because it reduces the danger of inadvertedly
starting the backup without recovery.

Writing it into the data directory is *not* a good thing because as soon as you do that you risk there being an issue if there’s a crash.  Writing into the backup isn’t a bad idea but if you’re managing your backups then writing it somewhere else (such as where you write your WAL) and associating it with the backup (presumably it has a label) should make it easy to pull back when you restore. 

> > > Lastly, if you really want, you can extract out the data from
> > > pg_stop_backup in whatever your post-backup script is.
> >
> > Come on, now.
> > You usually use backup techniques like that because you can't get
> > your large database backed up in the available time window otherwise.
>
> I’m not following what you’re trying to get at here, why can’t you extract
> the data for the backup label from pg_stop_backup..?  Certainly other tools
> do, even ones that do extremely fast parallel backups..  the two are
> completely independent.
>
> Did you think I meant pg_basebackup..?  I certaily didn’t.

Oh yes, I misunderstood.  Sorry.

Yes, you can come up with a post-backup script that somehow communicates
with your pre-backup script to get the information, but it sure is
inconvenient.  Simplicity is good in backup solutions, because complicated
things tend to break more easily.

Not sure what communication is necessary here..?   The data needed for the backup label file comes from pg_stop_backup in a non-exclusive backup.  You *should* be grabbing the starting WAL from the start backup as well, to confirm that you have all of the WAL for the backup, but you don’t actually need to in order to write out the backup label.

> > I thought our goal is to provide convenient backup methods...
>
> Correctness would be first and having a broken system because of a crash during a backup isn’t correct.

"Not starting up without manual intervention" is not actually broken...

Of course it is.  Having the behavior of the system be completely different depending on if a backup happened to be running or not is just plain bad and is broken.  It’s not a feature. Had this issue been realized when the exclusive backup mode was developed I suspect it never would have been implemented that way to begin with...

Thanks!

Stephen

В списке pgsql-hackers по дате отправления:

Предыдущее
От: Magnus Hagander
Дата:
Сообщение: Re: Updated backup APIs for non-exclusive backups
Следующее
От: Thomas Munro
Дата:
Сообщение: Re: [HACKERS] advanced partition matching algorithm forpartition-wise join