Hello All,
I have a 4 node PostgreSQL 9.6 cluster with streaming replication. we encounter today the Out of Memory Error on the Master which resulted in All postres processes restarted and cluster recovered itself. Please let me know the best way to diagnose this issue.
The error seen in the postgresql log:
2019-02-12 10:55:17 GMT LOG: terminating any other active server processes
2019-02-12 10:55:17 GMT WARNING: terminating connection because of crash of another server process
2019-02-12 10:55:17 GMT DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2019-02-12 10:55:17 GMT HINT: In a moment you should be able to reconnect to the database and repeat your command.
2019-02-12 10:55:17 GMT WARNING: terminating connection because of crash of another server process
2019-02-12 10:55:17 GMT DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2019-02-12 10:55:17 GMT HINT: In a moment you should be able to reconnect to the database and repeat your command.
2019-02-12 10:55:17 GMT WARNING: terminating connection because of crash of another server process
-----
Error from dmesg on linux:
-----------------------------------
[4331093.885622] Out of memory: Kill process nnnnn (postmaster) score nn or sacrifice child
[4331093.890225] Killed process nnnnn (postmaster) total-vm:18905944kB, anon-rss:1747460kB, file-rss:4kB, shmem-rss:838220kB
Thanks & Best Regards
Vikas Sharma