Re: block-level incremental backup

Greetings,

* Andres Freund (andres@anarazel.de) wrote:
> > > Wow.  I have to admit that I feel completely opposite of that- I'd
> > > *love* to have an independent tool (which ideally uses the same code
> > > through the common library, or similar) that can be run to apply WAL.
> > >
> > > In other words, I don't agree that it's the server's problem at all to
> > > solve that, or, at least, I don't believe that it needs to be.
> >
> > I mean, I guess I'd love to have that if I could get it by waving a
> > magic wand, but I wouldn't love it if I had to write the code or
> > maintain it.  The routines for applying WAL currently all assume that
> > you have a whole bunch of server infrastructure present; that code
> > wouldn't run in a frontend environment, I think.  I wouldn't want to
> > have a second copy of every WAL apply routine that might have its own
> > set of bugs.
>
> I'll fight tooth and nail not to have a second implementation of replay,
> even if it's just portions.  The code we have is complicated and fragile
> enough, having a [partial] second version would be way worse.  There's
> already plenty improvements we need to make to speed up replay, and a
> lot of them require multiple execution threads (be it processes or OS
> threads), something not easily feasible in a standalone tool. And
> without the already existing concurrent work during replay (primarily
> checkpointer doing a lot of the necessary IO), it'd also be pretty
> unattractive to use any separate tool.

I agree that we don't want another implementation and that there's a lot
that we want to do to improve replay performance.  We've already got
frontend tools which work with multiple execution threads, so I'm not
sure I get the "not easily feasible" bit, and the argument about the
checkpointer seems largely related to that (as in- if we didn't have
multiple threads/processes then things would perform quite badly...  but
we can and do have multiple threads/processes in frontend tools today,
even in pg_basebackup).

You certainly bring up some good concerns though and they make me think
of other bits that would seem like they'd possibly be larger issues for
a frontend tool- like having a large pool of memory for cacheing (aka
shared buffers) the changes.  If what we're talking about here is *just*
replay though, without having the system available for reads, I wonder
if we might want a different solution there.

> Unless you just define the server binary as that "independent tool".

That's certainly an interesting idea.

> Which I think is entirely reasonable. With the 'consistent' and LSN
> recovery targets one already can get most of what's needed from such a
> tool, anyway.  I'd argue the biggest issue there is that there's no
> equivalent to starting postgres with a private socket directory on
> windows, and perhaps an option or two making it easier to start postgres
> in a "private" mode for things like this.

This would mean building in a way to do parallel WAL replay into the
server binary though, as discussed above, and it seems like making that
work in a way that allows us to still be available as a read-only
standby would be quite a bit more difficult.  We could possibly support
parallel WAL replay only when we aren't a replica but from the same
binary.  The concerns mentioned about making it easier to start PG in a
private mode don't seem too bad but I am not entirely sure that the
tools which want to leverage that kind of capability would want to have
to exec out to the PG binary to use it.

A lot of this part of the discussion feels like a tangent though, unless
I'm missing something.  The "WAL compression" tool contemplated
previously would be much simpler and not the full-blown WAL replay
capability, which would be left to the server, unless you're suggesting
that even that should be exclusively the purview of the backend?  Though
that ship's already sailed, given that external projects have
implemented it.  Having a library to provide that which external
projects could leverage would be nicer than having everyone write their
own version.

Thanks!

Stephen