On Sat, 17 May 2008, Ioannis Tambouras wrote:
> The archive command tests if the wal segment exists and is a file,
> but it does not check if the file is still being written.
That's because it doesn't have to; the archive command doesn't get called
until the writing is done.
> I don't have sources of pg_standby near me, but I remember in the
> C code checks for complete segment sizes.
That's on the receiving side, to make sure it's not trying to process
files that haven't finished copying to the standby yet. You don't have to
do any of that in the archive_command.
Anyway, back to the original question:
> archive_command = 'test ! -f /usr/local/wal_archive_local/%f && cp %p
> /usr/local/wal_archive_local/%f'
> archive files are then moved on master to standby every other minute:
> rsync -aq --remove-sent-files /usr/local/wal_archive_local/
> slave::wal_archive/
I don't see any mechanism here to keep rsync from copying over partial
files to the standby before they've finished copying to the
wal_archive_local directory. That's my guess for where the small files
are coming from, rsync before the cp is done. If you're going to buffer
in a transfer directory, you need some sort of test or locking to make
sure the file is complete with exactly 16MB before it gets rsync'd over.
I suspect no amount of poking at the standby will root out the issue
because it's happening on the primary.
--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD