Andres Freund <andres@2ndquadrant.com> writes: > On 2014-06-03 11:04:58 -0400, Tom Lane wrote:
>> My point is that having backups crash on an overflow doesn't really seem >> acceptable. IMO we need to reconsider the basebackup protocol and make >> sure we don't *need* to put values over 4GB into this field. Where's the >> requirement coming from anyway --- surely all files in PGDATA ought to be >> 1GB max?
> Fujii's example was logfiles in pg_log. But we allow to change the > segment size via a configure flag, so we should support that or remove > the ability to change the segment size...
What we had better do, IMO, is fix things so that we don't have a filesize limit in the basebackup format. After a bit of googling, I found out that recent POSIX specs for tar format include "extended headers" that among other things support member files of unlimited size [1]. Rather than fooling with partial fixes, we should make the basebackup logic use an extended header when the file size is over INT_MAX.
Yeah, pax seems to be the way to go. It's at least supported by GNU tar - is it also supported on say BSD, or other popular platforms? (The size extension in the general ustar format seems to be, so it would be a shame if this one is less portable)