Re: Remove Deprecated Exclusive Backup Mode

Greetings,

* Andres Freund (andres@anarazel.de) wrote:
> On 2019-02-25 08:42:02 -0500, Stephen Frost wrote:
> > * Andres Freund (andres@anarazel.de) wrote:
> > > On 2019-02-24 17:19:22 -0500, Stephen Frost wrote:
> > > > You say above that the new interface is unquestionably an improvement
> > >
> > > FWIW, I think we didn't design it even remotely as well as we ought to
> > > have. It was both a mistake to not offer a version of non-exclusive
> > > backups that works with a client connection (for integration with the
> > > TSMs of this world), and it was a mistake not to offer a commandline
> > > tool that does the non-exclusive pg/start backup.
> >
> > I'm not really following- did you mean to say "without a client
> > connection" above?
>
> Yes.

Ah, alright, that makes more sense then.

You might bring that up with the individual who wrote it.

> > We could still add some kind of commandline tool to help with the
> > non-exclusive pg start/stop backup
>
> That doesn't help, as long as the connection requirement is there. As
> explained elsewhere in this thread, a lot of larger external backup
> infrastructure just gives you a start/stop hook, and that makes using a
> persistent connection hard and fragile.

I tend to agree w/ Magnus in that I'm not sure that this is actually
as common these days as claimed.  That said, I'm not against someone
trying to implement something that doesn't require a connection, though
I understand from discussion with David and his comments elsewhere on
this thread that it isn't that simple a thing to do- and people would
have to migrate to whatever it is anyway.

> > but I'm not really sure exactly what that would look like and I
> > honestly don't have terribly much interest in spending resources on
> > implementing it since it would lack a great many features that we'd
> > then end up wanting to add on... and then we end up backing into
> > building yet another backup tool.
>
> Meh. You are proposing to remove a feature (even if a dangerous
> one). Requiring some minimal compat work to make htat more palpaple
> isn't crazy.  Nor does using backrest or whatnot do *ANYTHING* about the
> users of large company wide backup tools.

Yes, I'm supporting the proposal to remove a dangerous feature.  I
really don't think much more needs to be said than that.  If the feature
wasn't dangerous then I wouldn't be argueing for its removal.  I do feel
bad that pgbackrest hasn't got a solution already to this, but we've had
other priorities.  I do think we'll get there, eventually, but not any
time soon.  Being told that I have to implement another feature so that
we can remove the dangerous one is something I don't agree with.

> > > > I really, honestly, believe what we need to *stop* doing is trying to
> > > > drag along support for old/deprecated interfaces after we've introduced
> > > > new ones, thus avoiding these arguments and the time spent on them, and
> > > > the time spent dealing with implementing and documenting new APIs around
> > > > old ones.  The only thing it seems to unquestionably do is make us argue
> > > > about when we are going to *finally* rip out the old/deprecated API (or
> > > > interface, or whatever you want to call it).
> > >
> > > While I agree that we should remove non-exclusive backups, I VEHEMENTLY
> > > oppose the unconditionality of the above statement. Yes, it's annoying
> > > to keep interfaces around, but unnecessarily inflicting pain to everyone
> > > also is annoying.
> >
> > Alright, then how about we provide a bit of help for everyone who's got
> > a system built around recovery.conf today, instead of just completely
> > ripping that out?
>
> Oh, comeon. Obviously we're sometimes going to have to make breaking
> changes. But you're essentially arguing that we shouldn't provide compat
> where we can come up with a reasonable way to provide backward
> compatibility. And I think that's crazy and will hurt postgres.

This was discussed, *extensively*, previously on this list with a whole
lot of proposals trying to come up with easy ways to "fix" exclusive
backups and I don't really think that this thread has proposed anything
novel in that regard.  Basically, I just don't think it's that easy.
Perhaps I'm wrong, in which case, great, I'd be happy to review a simple
patch that solves it.

> > > That's not to say we shouldn't be better at announcing and then
> > > following through on the deprecation of old interfaces.
> >
> > We announce things have changed every year, and people have five years
> > from that point, during which we'll continue to support them and fix
> > bugs and deal with security issues, to make whatever changes they need
> > to for the new interface.
> >
> > The argument seems to be that people want new features but they don't
> > want to have to do any work to get those new features.
>
> I mean, duh? We can't make that happen all the time, but I don't
> understand why it's surprising to have that as *one* goal that's in
> conflict with other goals?  Your analysis also neglects that a lot of
> software will have to work across multiple supported versions of
> postgres (at the very least in prep for the migration, but also often
> longer) - by having a few versions were both interfaces work, that can
> be made *MUCH* less painful.

Having to support multiple APIs like this really is painful.

Also, I'm pretty well versed in software that has to support multiple PG
major versions across time, using specifically these APIs in fact.
Having both interfaces work at the same time didn't make that any less
painful (pgbackrest supported non-exclusive backup in the same version
as 9.6 support was added, and supports backing up PG back to 8.3).
Perhaps it would help some, but I tend to doubt it based both on that
experience and in the experience that there tends to be two times people
make changes- opportunistically when they realize something has been
deprecated, and when they're forced to because the old interface went
away, and the level of effort and time spent tends to be about the same,
all that's different is when it's done.

> > When is something too big of a change to make in a given major version
> > and we have to, instead, provide backwards compatibility for it?  What
> > is that policy?  How many releases do we have to support it for?  How do
> > we communicate that to our users, so that they know what they can depend
> > on being there across major releases and what they can't?
>
> I literally said that we should have more policy around this.

Great.  I was kinda hoping you might propose something by way of
answering those questions so that maybe we could actually get to a point
of having such a policy.

Thanks!

Stephen