I have a client who has streaming replication setup with the primary in one city and standby in another city both identical servers with Postgresql 9.6 on Windows Server 2012.
They have some network issues, which is causing the connection from the primary to standby to drop sometimes. And recently standby crashed with the following log. And it could not be restarted.
2017-07-18 09:21:13 UTC FATAL: invalid memory alloc request size 4148830208
2017-07-18 09:21:14 UTC LOG: startup process (PID 5608) exited with exit code 1
2017-07-18 09:21:14 UTC LOG: terminating any other active server processes
2017-07-18 09:21:14 UTC LOG: database system is shut down
Last entry from the pg_xlogdump shows the following
pg_xlogdump: FATAL: error in WAL record at D5/D1BD5FD0: unexpected pageaddr D1/E7BD6000 in log segment 00000000000000D5000000D1, offset 12410880
So my questions are, could an old WAL segment being resent through the network cause crash like this? Shouldn’t Postgresql be able to handle out of order WAL segments instead of just crashing?
And what would be the best way to recover the standby server? Resynching the entire database seems to be too time consuming.
Thanks in advance for any info.
-Seong