Re: post-freeze damage control

On 4/12/24 12:15, Robert Haas wrote:
> On Thu, Apr 11, 2024 at 5:48 PM David Steele <david@pgmasters.net> wrote:
>> But they'll try because it is a new pg_basebackup feature and they'll
>> assume it is there to be used. Maybe it would be a good idea to make it
>> clear in the documentation that significant tooling will be required to
>> make it work.
> 
> I don't agree with that idea. LOTS of what we ship takes a significant
> amount of effort to make it work. You may well need a connection
> pooler. You may well need a failover manager which may or may not be
> separate from your connection pooler. You need a backup tool. You need
> a replication management tool which may or may not be separate from
> your backup tool and may or may not be separate from your failover
> tool. You probably need various out-of-core connections for the
> programming languages you need. You may need a management tool, and
> you probably need a monitoring tool. Some of the tools you might
> choose to do all that stuff themselves have a whole bunch of complex
> dependencies. It's a mess.

The difference here is you *can* use Postgres without a connection 
pooler (I have many times) or failover (if downtime is acceptable) but 
most people would agree that you really *need* backup.

The backup tool should be clear and easy to use or misery will 
inevitably result. pg_basebackup is difficult enough to use and automate 
because it has no notion of a repository, no expiration, and no WAL 
handling just to name a few things. Now there is an even more advanced 
feature that is even harder to use. So, no, I really don't think this 
feature is practically usable by the vast majority of end users.

> Now, if someone were to say that we ought to talk about these issues
> in our documentation and maybe give people some ideas about how to get
> started, I would likely be in favor of that, modulo the small
> political problem that various people would want their solution to be
> the canonical one to which everyone gets referred. But I think it's
> wrong to pretend like this feature is somehow special, that it's
> somehow more raw or unfinished than tons of other things. I actually
> think it's significantly *better* than a lot of other things. If we
> add a disclaimer to the documentation saying "hey, this new
> incremental backup feature is half-finished garbage!", and meanwhile
> the documentation still says "hey, you can use cp as your
> archive_command," then we have completely lost our minds.

Fair point on cp, but that just points to an overall lack in our 
documentation and built-in backup/recovery tools in general.

> I also think that you're being more negative about this than the facts
> justify. As I said to several colleagues today, I *fully* acknowledge
> that you have a lot more practical experience in this area than I do,
> and a bunch of good ideas. I was really pleased to see you talking
> about how it would be good if these tools worked on tar files - and I
> completely agree, and I hope that will happen, and I hope to help in
> making that happen. I think there are a bunch of other problems too,
> only some of which I can guess at. However, I think saying that this
> feature is not realistically intended to be used by end-users or that
> they will not be able to do so is over the top, and is actually kind
> of insulting. 

It is not meant to be insulting, but I still believe it to be true. 
After years of working with users on backup problems I think I have a 
pretty good bead on what the vast majority of admins are capable of 
and/or willing to do. Making this feature work is pretty high above that 
bar.

If the primary motivation is to provide a feature that can be integrated 
with third party tools, as Tomas suggests, then I guess usability is 
somewhat moot. But you are insisting that is not the case and I just 
don't see it that way.

> There has been more enthusiasm for this feature on this
> mailing list and elsewhere than I've gotten for anything I've
> developed in years. And I don't think that's because all of the people
> who have expressed enthusiasm are silly geese who don't understand how
> terrible it is.

No doubt there is enthusiasm. It's a great feature to have. In 
particular I think the WAL summarizer is cool. But I do think the 
shortcomings are significant and that will become very apparent when 
people start to implement. The last minute effort to add COW support is 
an indication of problems that people will see in the field.

Further, I do think some less that ideal design decisions were made. In 
particular, I think sidelining manifests, i.e. making them optional, is 
not a good choice. This has led directly to the issue we see in [1]. If 
we require a manifest to make an incremental backup, why make it 
optional for combine?

This same design decision has led us to have "marker files" for 
zero-length files and unchanged files, which just seems extremely 
wasteful when these could be noted in the manifest. There are good 
reasons for writing everything out in a full backup, but for an 
incremental that can only be reconstructed using our tool the manifest 
should be sufficient.

Maybe all of this can be improved in a future release, along with tar 
reading, but none of those potential future improvements help me to 
believe that this is a user-friendly feature in this release.

Regards,
-David

---

[1] 
https://www.postgresql.org/message-id/flat/9badd24d-5bd9-4c35-ba85-4c38a2feb73e%40pgmasters.net