On 4/12/24 11:50, David Steele wrote:
> On 4/12/24 19:09, Magnus Hagander wrote:
>> On Fri, Apr 12, 2024 at 12:14 AM David Steele <david@pgmasters.net
>>
>> ...>>
>> > But yeah, having to keep the backups as expanded directories is
>> not
>> > great, I'd love to have .tar. Not necessarily because of the disk
>> space
>> > (in my experience the compression in filesystems works quite
>> well for
>> > this purpose), but mostly because it's more compact and allows
>> working
>> > with backups as a single piece of data (e.g. it's much cleared
>> what the
>> > checksum of a single .tar is, compared to a directory).
>>
>> But again, object stores are commonly used for backup these days and
>> billing is based on data stored rather than any compression that
>> can be
>> done on the data. Of course, you'd want to store the compressed
>> tars in
>> the object store, but that does mean storing an expanded copy
>> somewhere
>> to do pg_combinebackup.
>>
>> Object stores are definitely getting more common. I wish they were
>> getting a lot more common than they actually are, because they
>> simplify a lot. But they're in my experience still very far from
>> being a majority.
>
> I see it the other way, especially the last few years. The majority seem
> to be object stores followed up closely by NFS. Directly mounted storage
> on the backup host appears to be rarer.
>
One thing I'd mention is that not having built-in support for .tar and
.tgz backups does not mean it's impossible to use pg_combinebackup with
archives. You can mount them using e.g. "ratarmount" and then use that
as source directories for pg_combinebackup.
It's not entirely friction-less because AFAICS it's necessary to do the
backup in plain format and then do the .tar to have the expected "flat"
directory structure (and not manifest + 2x tar). But other than that it
seems to work fine (based on my limited testing).
FWIW the "archivemount" performs terribly, so adding this capability
into pg_combinebackup is clearly far from trivial.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company