All was fine until:
LOG: statement: select "_devel".cleanupEvent('10 minutes'::interval,
'false'::boolean);
ERROR: could not open file "base/16406/2072097_fsm": Permission denied
STATEMENT: select "_devel".cleanupEvent('10 minutes'::interval,
'false'::boolean);
WARNING: AbortTransaction while in COMMIT state
PANIC: cannot abort transaction 7108311, it was already committed
LOG: unexpected EOF on client connection
LOG: unexpected EOF on client connection
LOG: unexpected EOF on client connection
LOG: server process (PID 21090) was terminated by signal 6: Aborted
LOG: terminating any other active server processes
WARNING: terminating connection because of crash of another server process
DETAIL: The postmaster has commanded this server process to roll back
the current transaction and exit, because another server process exited
abnormally and possibly corrupted shared memory.
...
DETAIL: The postmaster has commanded this server process to roll back
the current transaction and exit, because another server process exited
abnormally and possibly corrupted shared memory.
HINT: In a moment you should be able to reconnect to the database and
repeat your command.
LOG: all server processes terminated; reinitializing
LOG: database system was interrupted; last known up at 2011-12-07
14:51:37 EST
LOG: database system was not properly shut down; automatic recovery in
progress
LOG: consistent recovery state reached at 5/C9A28BE4
LOG: redo starts at 5/C9A26DBC
FATAL: could not open file "base/16406/2072097_fsm": Permission denied
CONTEXT: xlog redo commit: 2011-12-07 14:56:04.305309-05; rels:
base/16406/2072101 base/16406/2072100 base/16406/2072099
base/16406/2072098 base/16406/2072097
LOG: startup process (PID 25977) exited with exit code 1
LOG: aborting startup due to startup process failure
I was unable to restart pg until I did a chmod u+rw on that file. There
was nothing interesting in the log before the crash, just a bunch of
activity that was obviously slony.
version is:
PostgreSQL 9.1.2 on i686-pc-linux-gnu, compiled by gcc (GCC) 4.1.2
20080704 (Red Hat 4.1.2-48), 32-bit