Re: Remove Deprecated Exclusive Backup Mode

Greetings,

* Adrien NAYRAT (adrien.nayrat@anayrat.info) wrote:
> On 2/22/19 7:10 PM, Robert Haas wrote:
> >On Fri, Feb 22, 2019 at 12:35 PM Fujii Masao <masao.fujii@gmail.com> wrote:
> >>-1 for the removal. I think that there are still many users of an exclusive
> >>backup API, and it's not so easy to migrate their backup scripts so that
> >>only non-exclusive one is used because of the reason I wrote in another thread.
> >>https://postgr.es/m/CAHGQGwHUkEbkVexVfWNLjmq2rzOS_SHYMiECt+KBn-cBPq5Arg@mail.gmail.com
> >
> >Yeah, those are interesting points.  Unfortunately, I think something
> >as simple as your proposed...
> >
> >psql -c "SELECT pg_start_backup()"
> >rsync, cp, tar, storage backup, or something
> >psql -c "SELECT pg_stop_backup()"
> >
> >...doesn't have much chance of being correct.  If you aren't checking
> >whether pg_start_backup() throws an error, for example, you have got a
> >very failure-prone set-up.  That being said, it's possible to fix that
> >problem using some shell logic, whereas the problem of keeping a
> >connection open for the duration of the backup from the shell is much
> >harder.  I recently ran across a case where someone attempted it but
> >did not get the logic entirely right, with rather painful
> >consequences.
> >
> >Now you might say that people should not write their own tools and
> >just use professionally produced ones.  But I don't really agree with
> >that, and whatever we think people SHOULD do, there are in fact a lot
> >of people using their own tools.
>
> FWIW +1 for not remove exclusive backup.
>
> Maintain a connection during the backup is a hard pre-requisite. In my
> previous jobs I already done custom scripts to perform backup by using
> pre/post backup hook to control backup software: With vmware to do PITR
> backup with VM snapshot or with another file backup tool which perform block
> deduplication. I do not see where is the problem if you check
> pg_start_backup()/pg_stop_backup() went well?

This, really, is a large part of the problem- people don't realize and
just simply don't want to believe, it seems at least, that there's any
problem with what they're doing because it works some, or even most, of
the time.

Making a running PG cluster *look* like it was restored from a backup is
really bad and leads to a lot of confusion and cases where PG won't
start up, but that's what the exclusive pg_start_backup/pg_stop_backup
method does and is more-or-less *required* of any solution that wants to
just be able to tar up the data directory as the backup method, which is
why it ends up just being a bad interface overall.

And then there's other issues- things like making sure that you actually
got all of the WAL, without which the backup is inconsistent being the
leading one but there's others.

> It will be annoying if after this removal, companies must change their
> backup strategy by using specific postgres tools (pgbackrest, barman).

You don't write your own database system using CSV files and shell
magic, do you?  I have to say that it continues to boggle my mind that
people insist that *this* part of the system has to be able to be
implementable using shell scripts.

Folks, these are your backups we're talking about, your last resort if
everything else goes up in flames, why do you want to risk that by
implementing your own one-off solution, particularly when there's known
serious issues using that interface, and you want to just use shell
scripts to do it...?

Thanks!

Stephen