[HACKERS] possible self-deadlock window after bad ProcessStartupPacket

Hello,

There may possibly be a very small window for a double exit() self-deadlock during a forked backend process's ProcessStartupPacket returns and status is not STATUS_OK.  The process will go into proc_exit and then a very well timed SIGQUIT will call startup_die for another proc_exit.  If timed correctly, the two exit() calls can deadlock since exit() is not re-entrant.  It seems extremely hard to hit the deadlock but I believe the opportunity is there.

Using gdb, I was able to create the window and get this stacktrace:

#0  startup_die (postgres_signal_arg=0) at postmaster.c:5090
#1  <signal handler called>
#2  proc_exit_prepare (code=0) at ipc.c:158
#3  0x00000000007c4135 in proc_exit (code=0) at ipc.c:102
#4  0x000000000076b736 in BackendInitialize (port=0x2c13740) at postmaster.c:4207
#5  0x000000000076b190 in BackendStartup (port=0x2c13740) at postmaster.c:3979
#6  0x0000000000767ad3 in ServerLoop () at postmaster.c:1722
#7  0x00000000007671df in PostmasterMain (argc=3, argv=0x2bebad0) at postmaster.c:1330
#8  0x00000000006b5df6 in main (argc=3, argv=0x2bebad0) at main.c:228

I got the stacktrace by doing the following:

1. gdb attach to postmaster and run set follow-fork child and break postmaster.c:4206(right after ProcessStartupPacket) and continue
2. In another terminal, open a psql session which should trigger the gdb follow
3. In the gdb session, set status=1 and step into proc_exit()
4. In another terminal, kill -s QUIT <child pid> to send SIGQUIT to the child process. Or run pg_ctl stop -M immediate.
   
5. In the gdb session, step to process the signal into startup_die and run bt


This was discovered while hacking on Greenplum Database (currently based off of Postgres 8.3) where we recently started encountering the self-deadlock intermittently in our testing environment.

Here's the pull request discussion:
https://github.com/greenplum-db/gpdb/pull/1662

In that pull request, we fix the issue by checking for proc_exit_inprogress.  Is there a reason why startup_die should not check for proc_exit_inprogress?

In the above pull request, Heikki also mentions that a similar scenario can happen during palloc() as well... which is similar to what we saw in Greenplum a couple years back for a deadlock in a malloc() call where we responded by changing exit() to _exit() in quickdie as a fix.  That could possibly be applicable to latest Postgres as well.