Re: Improving compressibility of WAL files

 
> You don't want to just 
> modify pg_standby to accept small files, because then you've made it 
> harder to make absolutely sure when the file is ready to be 
> processed if a non-atomic copy is being done.

It is hard, but I think it is the right way forward.
Anyway I think the size is not robust at all because some (most ? e.g. win32) non-atomic copy
implementations will also show the final size right from the beginning.

Could we use stricter file locking when opening WAL for recovery ?

Or implement a wait loop when the crc check (+ a basic validity check) for the next
record fails (and the next record is on a 512 byte boundary ?).
I think standby and restore recovery can be treated differently to startup recovery because
a copied wal file, even if the copy is not atomic, will not have trailing valid WAL records
from a recycled WAL. (A solution that recycles WAL files for restore/standby would need to make
sure it renames the files *after* restoring the content.)

Btw how do we detect end of WAL when restoring a backup and WAL after PANIC ?

> 1) Provide the length as part of the archive command

+1

Andreas