Hi all,
A few days ago one of our postgres (8.3.12) servers was a bit unhappy, and someone decided to try a kill -9 on a
backendprocess after a kill (TERM) was ineffective. I've read many times in the past that a kill -9 can be pretty
hazardousto a postgres' health, and now it seems I get to see first hand how hazardous it really is :(
Fortunately postgres seems to have detected the -9 signal and brought the system down:
2011-08-05 17:17:53 EST redacted.com 10.3.0.3(39556) admin@redacted.com WARNING: terminating connection because of
crashof another server process
2011-08-05 17:17:53 EST redacted.com 10.3.0.3(39556) admin@redacted.com DETAIL: The postmaster has commanded this
serverprocess to roll back the current transaction and exit, because another server process exited abnormally and
possiblycorrupted shared memory.
2011-08-05 17:17:53 EST redacted.com 10.3.0.3(39556) admin@redacted.com HINT: In a moment you should be able to
reconnectto the database and repeat your command.
After the barrage of those messages, there:
2011-08-05 17:17:54 EST LOG: all server processes terminated; reinitializing
2011-08-05 17:17:55 EST LOG: database system was interrupted; last known up at 2011-08-05 17:15:33 EST
2011-08-05 17:17:55 EST LOG: database system was not properly shut down; automatic recovery in progress
2011-08-05 17:17:55 EST LOG: redo starts at 208/5013A758
2011-08-05 17:17:55 EST LOG: record with zero length at 208/51497498
2011-08-05 17:17:55 EST LOG: redo done at 208/51497468
2011-08-05 17:17:55 EST LOG: last completed transaction was at log time 2011-08-05 17:17:52.709539+10
2011-08-05 17:18:03 EST LOG: autovacuum launcher started
2011-08-05 17:18:03 EST LOG: database system is ready to accept connections
For each of the other backend processes.
I'm a bit worried about corruption and would like to know:
- Is postgres 8.3.12 susceptible to corruption when a backend process is -9'd?
- How do we confirm that there has been no corruption?
We have nightly backups that dump every database in the cluster, and looking over postgres' logs I can't see any errors
thatmight point to corruption... I guess that's a good sign - is there anything else I can look in to?
Thanks very much,
--Royce