Re: backup manifests

Hi,

On 2020-03-29 21:23:06 -0400, David Steele wrote:
> On 3/29/20 9:07 PM, Andres Freund wrote:
> > On 2020-03-29 20:42:35 -0400, Robert Haas wrote:
> > > > What do you think of having the verification process also call pg_waldump to
> > > > validate the WAL CRCs (shown upthread)?  That looked helpful and simple.
> > > 
> > > I don't love calls to external binaries, but I think the thing that
> > > really bothers me is that pg_waldump is practically bound to terminate
> > > with an error, because the last WAL segment will end with a partial
> > > record.
> > 
> > I don't think that's the case here. You should know the last required
> > record, which should allow to specify the precise end for pg_waldump. If
> > it errors out reading to that point, we'd be in trouble.
> 
> Exactly. All WAL generated during the backup should read fine with
> pg_waldump or there is a problem.

See the attached minimal prototype for what I am thinking of.

This would not correctly handle the case where the timeline changes
while taking a base backup. But I'm not sure that'd be all that serious
a limitation for now?

I'd personally not want to use a base backup that included a timeline
switch...

Greetings,

Andres Freund