Re: BUG #15036: Un-killable queries Hanging in BgWorkerShutdown

Sorry for the delay on this.

You can get a backtrace from a running program with by connecting to
it with gdb -p PID, then bt for the backtrace.  You might need to
install the symbols package if you only see addresses (on debianoid
systems postgresql-10-dbgsym, not sure what it's called on RHELish
systems).

https://wiki.postgresql.org/wiki/Getting_a_stack_trace_of_a_running_PostgreSQL_backend_on_Linux/BSD

here's a backtrace from one of the running pids:  

```

#0  0x00007f699037c9f3 in __epoll_wait_nocancel () at ../sysdeps/unix/syscall-template.S:84

#1  0x00005639b9d28c61 in WaitEventSetWaitBlock (nevents=1, occurred_events=0x7ffd43678d90, cur_timeout=-1, set=0x5639bbef83c8) at /build/postgresql-10-drhiey/postgresql-10-10.3/build/../src/backend/storage/ipc/latch.c:1048

#2  WaitEventSetWait (set=set@entry=0x5639bbef83c8, timeout=timeout@entry=-1, occurred_events=occurred_events@entry=0x7ffd43678d90, nevents=nevents@entry=1, wait_event_info=wait_event_info@entry=134217728)

    at /build/postgresql-10-drhiey/postgresql-10-10.3/build/../src/backend/storage/ipc/latch.c:1000

#3  0x00005639b9d290d4 in WaitLatchOrSocket (latch=0x7f697e506f54, wakeEvents=wakeEvents@entry=17, sock=sock@entry=-1, timeout=-1, timeout@entry=0, wait_event_info=wait_event_info@entry=134217728)

    at /build/postgresql-10-drhiey/postgresql-10-10.3/build/../src/backend/storage/ipc/latch.c:385

#4  0x00005639b9d29185 in WaitLatch (latch=<optimized out>, wakeEvents=wakeEvents@entry=17, timeout=timeout@entry=0, wait_event_info=wait_event_info@entry=134217728)

    at /build/postgresql-10-drhiey/postgresql-10-10.3/build/../src/backend/storage/ipc/latch.c:339

#5  0x00005639b9cccf1b in WaitForBackgroundWorkerShutdown (handle=0x5639bbe8aaf0) at /build/postgresql-10-drhiey/postgresql-10-10.3/build/../src/backend/postmaster/bgworker.c:1154

#6  0x00005639b9af36fd in WaitForParallelWorkersToExit (pcxt=0x5639bbe8a118, pcxt=0x5639bbe8a118) at /build/postgresql-10-drhiey/postgresql-10-10.3/build/../src/backend/access/transam/parallel.c:655

#7  0x00005639b9af4417 in DestroyParallelContext (pcxt=0x5639bbe8a118) at /build/postgresql-10-drhiey/postgresql-10-10.3/build/../src/backend/access/transam/parallel.c:737

#8  0x00005639b9af4a28 in AtEOXact_Parallel (isCommit=isCommit@entry=0 '\000') at /build/postgresql-10-drhiey/postgresql-10-10.3/build/../src/backend/access/transam/parallel.c:1006

#9  0x00005639b9affde7 in AbortTransaction () at /build/postgresql-10-drhiey/postgresql-10-10.3/build/../src/backend/access/transam/xact.c:2538

#10 0x00005639b9b00545 in AbortCurrentTransaction () at /build/postgresql-10-drhiey/postgresql-10-10.3/build/../src/backend/access/transam/xact.c:3097

#11 0x00005639b9d4bd6d in PostgresMain (argc=1, argv=argv@entry=0x5639bbe9ae40, dbname=0x5639bbe9ad58 "marjory", username=0x5639bbe39a08 "reporter") at /build/postgresql-10-drhiey/postgresql-10-10.3/build/../src/backend/tcop/postgres.c:3879

#12 0x00005639b9a850d9 in BackendRun (port=0x5639bbe978f0) at /build/postgresql-10-drhiey/postgresql-10-10.3/build/../src/backend/postmaster/postmaster.c:4405

#13 BackendStartup (port=0x5639bbe978f0) at /build/postgresql-10-drhiey/postgresql-10-10.3/build/../src/backend/postmaster/postmaster.c:4077

#14 ServerLoop () at /build/postgresql-10-drhiey/postgresql-10-10.3/build/../src/backend/postmaster/postmaster.c:1755

#15 0x00005639b9cdb78b in PostmasterMain (argc=5, argv=<optimized out>) at /build/postgresql-10-drhiey/postgresql-10-10.3/build/../src/backend/postmaster/postmaster.c:1363

#16 0x00005639b9a864d5 in main (argc=5, argv=0x5639bbe37850) at /build/postgresql-10-drhiey/postgresql-10-10.3/build/../src/backend/main/main.c:228

```

Joe, cc'd here is a colleague who will be able to help out in future. 

Thanks for all the help on this,

David