OK, starting over...
I killed off both standby servers and I rebuilt my databases off of the primary server (10.10.1.128) using:
pg_basebackup -h 10.10.1.128 -p 5432 -D /var/lib/pgsql/9.3/data -X s -c fast
It took about 40 minutes to complete for each standby. Immediately after it completed, I copied my previously saved
recovery.conffile into each's data directory and started the
standby servers. I did not do anything to the files in the directory specified by the restore_command. Now, I would
expectthe primary to continue writing WAL segments into the
directory while the pg_basebackup is running, but it apparently did not. Checking the primary's pg_xlog directory
showedthat it was now writing *DF, and each standby had only
received up to *D5. That means each standby's WALs are about 1 hour behind. I would expect each to be at *DE as that
shouldbe the last completely written WAL file, and I would
guess that the primary is still writing *DF.
The archive process on the primary is now writing WAL segments to the directory in archive_command, as I just saw *D6
popinto it, but it's never going to catch up without help as
it's only copying 1 file every 10 minutes as a new one gets generated. Am I missing a flag for pg_basebackup? Or should
Ijust accept that the standby's could never catch up with
WAL segments and just turn archiving off?
--
Jay