COPY process hung in 9.2.x (9.2.4)

Поиск
Список
Период
Сортировка
От mark
Тема COPY process hung in 9.2.x (9.2.4)
Дата
Msg-id CAKD=pjjCbC9a9AdqwrKRRQJNAhKgD+JO3zQN0Xr9tK6q8Xgwbw@mail.gmail.com
обсуждение исходный текст
Ответы Re: COPY process hung in 9.2.x (9.2.4)  (Tom Lane <tgl@sss.pgh.pa.us>)
Список pgsql-bugs
Hi all,

I have been seeing some copy jobs hang in a odd way. I am unable to
repoduce on demand, but here is what I have.


my setup is a bit complex, but should be old hat for most people.

client connection runs through bouncer (session pooling) to a 9.2.4
postgres back end. I have had this problem happen without bouncer in the
middle but for this limited debug bouncer is in play. have numerous clinets
copying to the same table throughout the day, often concurrently.


this is not the first time I have a copy job that won't quit/die from a
pg_cancel_backend or a pg_terminate backend.


this copy hang issue only stated since upgrading to 9.2 (from 8.3)


it looks to my untrained eye that it's in some wait state that can't be
interrupted?.



here is what the process looks like in pg_stat_activity;

I took this at about 2013-07-17 14:30-06 so the process had been in a funky
state for a number of hours.



-[ RECORD 1
]----+---------------------------------------------------------------------=
---------------------------------------------------------------------------=
---------------------------------------------------------------------------=
-----------------------------------------------
datid            | 16384
datname          | XXXXXXX
pid              | 25896
usesysid         | 10
usename          | postgres
application_name |
client_addr      | 10.1.106.131
client_hostname  |
client_port      | 60875
backend_start    | 2013-07-08 14:36:37.223981-06
xact_start       | 2013-07-17 08:36:33.139472-06
query_start      | 2013-07-17 08:36:33.139472-06
state_change     | 2013-07-17 08:36:33.139474-06
waiting          | f
state            | active
query            | COPY  (<snip>)


XXXXXXX=3D# SELECT pg_cancel_backend(25896);
 pg_cancel_backend
-------------------
 t
(1 row)

XXXXXXX=3D# SELECT pg_cancel_backend(25896);
 pg_cancel_backend
-------------------
 t
(1 row)

XXXXXXX=3D# SELECT pg_cancel_backend(25896);
 pg_cancel_backend
-------------------
 t
(1 row)

XXXXXXX=3D# SELECT pg_cancel_backend(25896);
 pg_cancel_backend
-------------------
 t
(1 row)

XXXXXXX=3D#SELECT pg_terminate_backend(25896);
 pg_terminate_backend
----------------------
 t
(1 row)

XXXXXXX=3D# SELECT pg_terminate_backend(25896);
 pg_terminate_backend
----------------------
 t
(1 row)




ok... so it will send the signal, just not get around to acting on it.


gdb looked unhelpul since there are no debugging symbols in the PGDG RPMS.
(which I would not expect there to be)


[root@]# gstack 25896
#0  0x000000317ece9672 in recv () from /lib64/libc.so.6
#1  0x00000000005a9676 in secure_read ()
#2  0x00000000005b432b in ?? ()
#3  0x00000000005b465c in pq_getmessage ()
#4  0x000000000052cdaa in ?? ()
#5  0x000000000052d297 in ?? ()
#6  0x000000000052d3c6 in ?? ()
#7  0x000000000052dca7 in NextCopyFromRawFields ()
#8  0x000000000052e5b9 in NextCopyFrom ()
#9  0x0000000000531b59 in DoCopy ()
#10 0x000000000065dbeb in standard_ProcessUtility ()
#11 0x000000000065a607 in ?? ()
#12 0x000000000065b5fd in ?? ()
#13 0x000000000065bd82 in PortalRun ()
#14 0x000000000065834d in ?? ()
#15 0x0000000000659200 in PostgresMain ()
#16 0x0000000000615161 in ?? ()
#17 0x0000000000617dcc in PostmasterMain ()
#18 0x00000000005b6830 in main ()


strace showed that nothing was going on for several minutes:

 strace -p 25896
Process 25896 attached - interrupt to quit
recvfrom(10, ^C <unfinished ...>
Process 25896 detached





ps -efl | grep 25896
1 S postgres 25896 24852  0  80   0 - 604148 sk_wai Jul08 ?       00:00:11
postgres: postgres XXXXXXX 10.1.106.131(60875) COPY




lsof -n -p 25896
COMMAND    PID     USER   FD   TYPE   DEVICE   SIZE/OFF       NODE NAME
postgres 25896 postgres  cwd    DIR      8,1       4096        131
/raid/data92
postgres 25896 postgres  rtd    DIR    253,0       4096          2 /
postgres 25896 postgres  txt    REG    253,0    5477296    7479113
/usr/pgsql-9.2/bin/postgres
postgres 25896 postgres  mem    REG    253,0     156872     524290 /lib64/
ld-2.12.so
postgres 25896 postgres  mem    REG    253,0    1922152     524294 /lib64/
libc-2.12.so
postgres 25896 postgres  mem    REG    253,0      22536     524317 /lib64/
libdl-2.12.so
postgres 25896 postgres  mem    REG    253,0     145720     524295 /lib64/
libpthread-2.12.so
postgres 25896 postgres  mem    REG    253,0      91096     524428
/lib64/libz.so.1.2.3
postgres 25896 postgres  mem    REG    253,0     598680     524315 /lib64/
libm-2.12.so
postgres 25896 postgres  mem    REG    253,0     124624     524372
/lib64/libselinux.so.1
postgres 25896 postgres  mem    REG    253,0     113952     524376 /lib64/
libresolv-2.12.so
postgres 25896 postgres  mem    REG    253,0      43392     524349 /lib64/
libcrypt-2.12.so
postgres 25896 postgres  mem    REG    253,0     386040     524342
/lib64/libfreebl3.so
postgres 25896 postgres  mem    REG    253,0     108728    7475210
/usr/lib64/libsasl2.so.2.0.23
postgres 25896 postgres  mem    REG    253,0      17256     524441
/lib64/libcom_err.so.2.1
postgres 25896 postgres  mem    REG    253,0      58480     524352
/lib64/libpam.so.0.82.2
postgres 25896 postgres  mem    REG    253,0      17096     524419
/lib64/libplds4.so
postgres 25896 postgres  mem    REG    253,0    1286744    7479035
/usr/lib64/libnss3.so
postgres 25896 postgres  mem    REG    253,0     177952    7475272
/usr/lib64/libnssutil3.so
postgres 25896 postgres  mem    REG    253,0     224328    7479710
/usr/lib64/libssl3.so
postgres 25896 postgres  mem    REG    253,0      12592     524438
/lib64/libkeyutils.so.1.3
postgres 25896 postgres  mem    REG    253,0     243096     524388
/lib64/libnspr4.so
postgres 25896 postgres  mem    REG    253,0    1665328    7479326
/usr/lib64/libcrypto.so.1.0.0
postgres 25896 postgres  mem    REG    253,0     183896    7479978
/usr/lib64/libsmime3.so
postgres 25896 postgres  mem    REG    253,0    1383336    7474856
/usr/lib64/libxml2.so.2.7.6
postgres 25896 postgres  mem    REG    253,0     944712     524324
/lib64/libkrb5.so.3.3
postgres 25896 postgres  mem    REG    253,0     177520     524310
/lib64/libk5crypto.so.3.1
postgres 25896 postgres  mem    REG    253,0     280520     524330
/lib64/libgssapi_krb5.so.2.2
postgres 25896 postgres  mem    REG    253,0      46368     524304
/lib64/libkrb5support.so.0.1
postgres 25896 postgres  mem    REG    253,0      60144    7474481
/usr/lib64/liblber-2.4.so.2.8.5
postgres 25896 postgres  mem    REG    253,0     383608    7480990
/usr/lib64/libssl.so.1.0.0
postgres 25896 postgres  mem    REG    253,0     326216    7480986
/usr/lib64/libldap-2.4.so.2.8.5
postgres 25896 postgres  mem    REG    253,0      65928     524589 /lib64/
libnss_files-2.12.so
postgres 25896 postgres  DEL    REG      0,4                 65536
/SYSV0052e2c1
postgres 25896 postgres  mem    REG    253,0   99158576    7474747
/usr/lib/locale/locale-archive
postgres 25896 postgres  mem    REG    253,0      21256     524417
/lib64/libplc4.so
postgres 25896 postgres  mem    REG    253,0     115536     524329
/lib64/libaudit.so.1.0.0
postgres 25896 postgres    0r   CHR      1,3        0t0       3842 /dev/nul=
l
postgres 25896 postgres    1w  FIFO      0,8        0t0    2353970 pipe
postgres 25896 postgres    2w  FIFO      0,8        0t0    2353970 pipe
postgres 25896 postgres    3r  FIFO      0,8        0t0   51158748 pipe
postgres 25896 postgres    4w  FIFO      0,8        0t0   51158748 pipe
postgres 25896 postgres    5u   REG      8,1      16384 2147483827
/raid/data92/global/12619
postgres 25896 postgres    6r  FIFO      0,8        0t0    2353969 pipe
postgres 25896 postgres    7u   REG    253,0   16777216    7078308
/var/xlogs/pg_xlog92/000000010000280D0000006A (deleted)
postgres 25896 postgres    8u   REG      8,1      16384 2147483828
/raid/data92/global/12620
postgres 25896 postgres    9u  IPv6  2353978        0t0        UDP
[::1]:34550->[::1]:34550
*postgres 25896 postgres   10u  IPv4 51158747        0t0        TCP
10.1.106.131:postgres->10.1.106.131:60875 (ESTABLISHED)*
postgres 25896 postgres   11u   REG      8,1      16384 2147483785
/raid/data92/global/12868
postgres 25896 postgres   12u   REG      8,1     344064 2147489656
/raid/data92/base/16384/247678053_fsm
postgres 25896 postgres   13u   REG      8,1      16384 2147483787
/raid/data92/global/12869
postgres 25896 postgres   14u   REG      8,1    1048576 2147483873
/raid/data92/base/16384/12636
postgres 25896 postgres   15u   REG      8,1 1073741824 2147488687
/raid/data92/base/16384/247678053
postgres 25896 postgres   16u   REG      8,1     180224 2147490208
/raid/data92/base/16384/247666671_fsm
postgres 25896 postgres   17u   REG      8,1  255205376 2147490131
/raid/data92/base/16384/247678062
postgres 25896 postgres   18u   REG      8,1      24576 2147490213
/raid/data92/base/16384/247678802_fsm
postgres 25896 postgres   19u   REG      8,1      16384 2147486050
/raid/data92/base/16384/12675
postgres 25896 postgres   20u   REG      8,1   24592384 2147490148
/raid/data92/base/16384/247678802
postgres 25896 postgres   21u   REG      8,1      32768 2147483805
/raid/data92/base/16384/12687
postgres 25896 postgres   22u   REG      8,1    4784128 2147490223
/raid/data92/base/16384/247678811
postgres 25896 postgres   23u   REG      8,1       8192 2147483831
/raid/data92/global/12751
postgres 25896 postgres   24u   REG      8,1      16384 2147486062
/raid/data92/base/16384/12731
postgres 25896 postgres   25u   REG      8,1    1556480 2147490065
/raid/data92/base/16384/247660214_fsm
postgres 25896 postgres   26u   REG      8,1 1073741824 2147486178
/raid/data92/base/16384/247660214
postgres 25896 postgres   27u   REG      8,1 1073741824 2147491444
/raid/data92/base/16384/247660214.1
postgres 25896 postgres   28u   REG      8,1 1073741824 2147486188
/raid/data92/base/16384/247660219
postgres 25896 postgres   29u   REG      8,1 1073741824 2147491455
/raid/data92/base/16384/247660219.1
postgres 25896 postgres   30u   REG      8,1     106496 2147490211
/raid/data92/base/16384/247661998_fsm
postgres 25896 postgres   31u   REG      8,1  357679104 2147486635
/raid/data92/base/16384/247661998
postgres 25896 postgres   32u   REG      8,1      24576 2147490254
/raid/data92/base/16384/247678042_fsm
postgres 25896 postgres   33u   REG      8,1    2490368 2147488566
/raid/data92/base/16384/247678042
postgres 25896 postgres   34u   REG      8,1     278528 2147490210
/raid/data92/base/16384/247678848_fsm
postgres 25896 postgres   35u   REG      8,1 1034158080 2147490224
/raid/data92/base/16384/247678848
postgres 25896 postgres   36u   REG      8,1  564789248 2147490231
/raid/data92/base/16384/247678858
postgres 25896 postgres   37u   REG      8,1  150011904 2147490130
/raid/data92/base/16384/247678061
postgres 25896 postgres   38u   REG      8,1      57344 2147490212
/raid/data92/base/16384/247679246_fsm
postgres 25896 postgres   39u   REG      8,1  157581312 2147490232
/raid/data92/base/16384/247679246
postgres 25896 postgres   40u   REG      8,1  143163392 2147490235
/raid/data92/base/16384/247679254
postgres 25896 postgres   41u   REG      8,1    1998848 2147490055
/raid/data92/base/16384/247664756_fsm
postgres 25896 postgres   42u   REG      8,1 1073741824 2147488227
/raid/data92/base/16384/247664756
postgres 25896 postgres   43u   REG      8,1 1073741824 2147491477
/raid/data92/base/16384/247664756.1
postgres 25896 postgres   44u   REG      8,1 1073741824 2147491456
/raid/data92/base/16384/247664756.2
postgres 25896 postgres   45u   REG      8,1     409600 2147490122
/raid/data92/base/16384/247662229_fsm
postgres 25896 postgres   46u   REG      8,1  646856704 2147488314
/raid/data92/base/16384/247666671
postgres 25896 postgres   47u   REG      8,1    1343488 2147490090
/raid/data92/base/16384/247662566_fsm
postgres 25896 postgres   48u   REG      8,1 1073741824 2147486688
/raid/data92/base/16384/247662566
postgres 25896 postgres   49u   REG      8,1 1073741824 2147549657
/raid/data92/base/16384/247662566.1
postgres 25896 postgres   50u   REG      8,1     262144 2147490172
/raid/data92/base/16384/247664480_fsm
postgres 25896 postgres   51u   REG      8,1  972759040 2147488173
/raid/data92/base/16384/247664480
postgres 25896 postgres   52u   REG      8,1  441982976 2147488210
/raid/data92/base/16384/247664486
postgres 25896 postgres   53u   REG      8,1  722575360 2147488174
/raid/data92/base/16384/247664485
postgres 25896 postgres   54u   REG      8,1  712818688 2147486668
/raid/data92/base/16384/247662245
postgres 25896 postgres   55u   REG      8,1 1073741824 2147486665
/raid/data92/base/16384/247662229
postgres 25896 postgres   56u   REG      8,1 1073741824 2147486201
/raid/data92/base/16384/247660220
postgres 25896 postgres   57u   REG      8,1 1073741824 2147549688
/raid/data92/base/16384/247660220.1
postgres 25896 postgres   58u   REG      8,1      24576 2147490218
/raid/data92/base/16384/247679258_fsm
postgres 25896 postgres   59u   REG      8,1   27156480 2147490268
/raid/data92/base/16384/247679258
postgres 25896 postgres   60u   REG      8,1   82960384 2147490264
/raid/data92/base/16384/247679256
postgres 25896 postgres   61u   REG      8,1  101376000 2147490259
/raid/data92/base/16384/247679255
postgres 25896 postgres   62u   REG      8,1  421117952 2147490127
/raid/data92/base/16384/247678060
postgres 25896 postgres   63u   REG      8,1 1073741824 2147486694
/raid/data92/base/16384/247662571
postgres 25896 postgres   64u   REG      8,1  944750592 2147490228
/raid/data92/base/16384/247678856
postgres 25896 postgres   65u   REG      8,1  665083904 2147490230
/raid/data92/base/16384/247678857
postgres 25896 postgres   66u   REG      8,1 1073741824 2147488250
/raid/data92/base/16384/247664761
postgres 25896 postgres   67u   REG      8,1 1073741824 2147491453
/raid/data92/base/16384/247664761.1
postgres 25896 postgres   68u   REG      8,1 1073741824 2147486702
/raid/data92/base/16384/247662572
postgres 25896 postgres   69u   REG      8,1 1073741824 2147549740
/raid/data92/base/16384/247664756.4
postgres 25896 postgres   70u   REG      8,1 1073741824 2147549690
/raid/data92/base/16384/247660214.2
postgres 25896 postgres   71u   REG      8,1  185663488 2147486647
/raid/data92/base/16384/247662004
postgres 25896 postgres   72u   REG      8,1 1073741824 2147549703
/raid/data92/base/16384/247664756.3
postgres 25896 postgres   73u   REG      8,1 1073741824 2147488255
/raid/data92/base/16384/247664762
postgres 25896 postgres   74u   REG      8,1 1073741824 2147549679
/raid/data92/base/16384/247664762.1
postgres 25896 postgres   75u   REG      8,1     483328 2147486037
/raid/data92/base/16384/12639
postgres 25896 postgres   76u   REG      8,1     401408 2147483825
/raid/data92/base/16384/12638
postgres 25896 postgres   77u   REG      8,1    2203648 2147486033
/raid/data92/base/16384/12628
postgres 25896 postgres   78u   REG      8,1    9355264 2147483909
/raid/data92/base/16384/12625
postgres 25896 postgres   79u   REG      8,1     188416 2147486040
/raid/data92/base/16384/12645
postgres 25896 postgres   80u   REG      8,1    1261568 2147483884
/raid/data92/base/16384/12640
postgres 25896 postgres   81u   REG      8,1     204800 2147486047
/raid/data92/base/16384/12662
postgres 25896 postgres   82u   REG      8,1     630784 2147483934
/raid/data92/base/16384/12660
postgres 25896 postgres   83u   REG      8,1  254623744 2147488319
/raid/data92/base/16384/247666677
postgres 25896 postgres   84u   REG      8,1  327983104 2147486642
/raid/data92/base/16384/247662003
postgres 25896 postgres   85u   REG      8,1 1073741824 2147549709
/raid/data92/base/16384/247660219.2
postgres 25896 postgres   86u   REG      8,1 1073741824 2147549707
/raid/data92/base/16384/247664761.2
postgres 25896 postgres   87u   REG      8,1 1073741824 2147549696
/raid/data92/base/16384/247662571.1
postgres 25896 postgres   88u   REG      8,1 1073741824 2147549719
/raid/data92/base/16384/247662566.2
postgres 25896 postgres   89u   REG      8,1 1073741824 2147549730
/raid/data92/base/16384/247660214.3
postgres 25896 postgres  128u   REG      8,1     196608 2147486048
/raid/data92/base/16384/12663
postgres 25896 postgres  138u   REG      8,1      16384 2147486057
/raid/data92/base/16384/12705
postgres 25896 postgres  139u   REG      8,1     122880 2147483807
/raid/data92/base/16384/12699
postgres 25896 postgres  140u   REG      8,1     188416 2147483808
/raid/data92/base/16384/12703
postgres 25896 postgres  141u   REG      8,1     688128 2147483892
/raid/data92/base/16384/12701
postgres 25896 postgres  142u   REG      8,1     212992 2147486036
/raid/data92/base/16384/12635
postgres 25896 postgres  143u   REG      8,1     655360 2147483883
/raid/data92/base/16384/12629
postgres 25896 postgres  144u   REG      8,1      73728 2147486035
/raid/data92/base/16384/12634
postgres 25896 postgres  145u   REG      8,1     335872 2147486030
/raid/data92/base/16384/12616
postgres 25896 postgres  146u   REG      8,1      40960 2147483789
/raid/data92/base/16384/12667
postgres 25896 postgres  147u   REG      8,1      32768 2147483902
/raid/data92/base/16384/12666
postgres 25896 postgres  148u   REG      8,1      16384 2147483900
/raid/data92/base/16384/12826
postgres 25896 postgres  149u   REG      8,1     368640 2147483810
/raid/data92/base/16384/12615
postgres 25896 postgres  150u   REG      8,1      16384 2147486016
/raid/data92/base/16384/12723
postgres 25896 postgres  151u   REG      8,1     999424 2147483921
/raid/data92/base/16384/12613
postgres 25896 postgres  152u   REG      8,1      16384 2147483925
/raid/data92/base/16384/12674
postgres 25896 postgres  153u   REG      8,1      32768 2147483806
/raid/data92/base/16384/12682
postgres 25896 postgres  154u   REG      8,1      16384 2147486055
/raid/data92/base/16384/12698
postgres 25896 postgres  155u   REG      8,1      32768 2147486051
/raid/data92/base/16384/12683
postgres 25896 postgres  156u   REG      8,1      32768 2147483888
/raid/data92/base/16384/12680
postgres 25896 postgres  158u   REG      8,1    1073152 2147486029
/raid/data92/base/16384/12612
postgres 25896 postgres  161u   REG      8,1 1073741824 2147491089
/raid/data92/base/16384/245572574
postgres 25896 postgres  162u   REG      8,1 1073741824 2147486571
/raid/data92/base/16384/245572574.1
postgres 25896 postgres  163u   REG      8,1 1073741824 2147617642
/raid/data92/base/16384/245572574.2
postgres 25896 postgres  164u   REG      8,1 1073741824 2147548355
/raid/data92/base/16384/245572574.3
postgres 25896 postgres  165u   REG      8,1 1073741824 2147488185
/raid/data92/base/16384/245572574.4
postgres 25896 postgres  166u   REG      8,1 1005936640 2147491104
/raid/data92/base/16384/245572574.5
postgres 25896 postgres  167u   REG      8,1 1073741824 2147490062
/raid/data92/base/16384/248233035
postgres 25896 postgres  168u   REG      8,1 1073741824 2147490063
/raid/data92/base/16384/248233035.1
postgres 25896 postgres  169u   REG      8,1 1073741824 2147490070
/raid/data92/base/16384/248233035.2
postgres 25896 postgres  170u   REG      8,1   17883136 2147490072
/raid/data92/base/16384/248233035.3
postgres 25896 postgres  171u   REG      8,1 1073741824 2147490080
/raid/data92/base/16384/248233173
postgres 25896 postgres  172u   REG      8,1  832438272 2147490087
/raid/data92/base/16384/248233173.1
postgres 25896 postgres  173u   REG      8,1 1073741824 2147486172
/raid/data92/base/16384/238131664
postgres 25896 postgres  174u   REG      8,1 1073741824 2147486356
/raid/data92/base/16384/238131664.1
postgres 25896 postgres  175u   REG      8,1 1073741824 2147503855
/raid/data92/base/16384/238131664.2
postgres 25896 postgres  176u   REG      8,1 1073741824 2147548614
/raid/data92/base/16384/238131664.3
postgres 25896 postgres  177u   REG      8,1 1073741824 2147490347
/raid/data92/base/16384/238131664.4
postgres 25896 postgres  178u   REG      8,1 1073741824 2147489603
/raid/data92/base/16384/238131664.5
postgres 25896 postgres  179u   REG      8,1  317358080 2147548647
/raid/data92/base/16384/238131664.6
postgres 25896 postgres  180u   REG      8,1 1073741824 2147488447
/raid/data92/base/16384/248218634
postgres 25896 postgres  181u   REG      8,1 1073741824 2147488448
/raid/data92/base/16384/248218634.1
postgres 25896 postgres  182u   REG      8,1 1073741824 2147488450
/raid/data92/base/16384/248218634.2
postgres 25896 postgres  183u   REG      8,1  206184448 2147488453
/raid/data92/base/16384/248218634.3
postgres 25896 postgres  184u   REG      8,1 1073741824 2147488454
/raid/data92/base/16384/248218763
postgres 25896 postgres  185u   REG      8,1  943489024 2147488456
/raid/data92/base/16384/248218763.1
postgres 25896 postgres  186u   REG      8,1 1073741824 2147549775
/raid/data92/base/16384/236347143
postgres 25896 postgres  187u   REG      8,1 1073741824 2147503866
/raid/data92/base/16384/236347143.1
postgres 25896 postgres  188u   REG      8,1 1073741824 2147486568
/raid/data92/base/16384/236347143.2
postgres 25896 postgres  189u   REG      8,1 1073741824 2147747483
/raid/data92/base/16384/236347143.3
postgres 25896 postgres  190u   REG      8,1 1073741824 2147486232
/raid/data92/base/16384/236347143.4
postgres 25896 postgres  191u   REG      8,1  975126528 2147489633
/raid/data92/base/16384/236347143.5
postgres 25896 postgres  192u   REG      8,1 1073741824 2147616724
/raid/data92/base/16384/247489640
postgres 25896 postgres  193u   REG      8,1 1073741824 2147616728
/raid/data92/base/16384/247489640.1
postgres 25896 postgres  194u   REG      8,1 1071448064 2147616729
/raid/data92/base/16384/247489640.2
postgres 25896 postgres  195u   REG      8,1 1073741824 2147617609
/raid/data92/base/16384/247489709
postgres 25896 postgres  196u   REG      8,1  818839552 2147617613
/raid/data92/base/16384/247489709.1
postgres 25896 postgres  197u   REG      8,1 1073741824 2147499926
/raid/data92/base/16384/242785737
postgres 25896 postgres  198u   REG      8,1 1073741824 2147488719
/raid/data92/base/16384/242785737.1
postgres 25896 postgres  199u   REG      8,1 1073741824 2147878017
/raid/data92/base/16384/242785737.2
postgres 25896 postgres  200u   REG      8,1 1073741824 2147747169
/raid/data92/base/16384/242785737.3
postgres 25896 postgres  201u   REG      8,1 1073741824 2147747213
/raid/data92/base/16384/242785737.4
postgres 25896 postgres  202u   REG      8,1 1073741824 2147617717
/raid/data92/base/16384/242785737.5
postgres 25896 postgres  203u   REG      8,1  555130880 2147548558
/raid/data92/base/16384/242785737.6
postgres 25896 postgres  204u   REG      8,1 1073741824 2147488714
/raid/data92/base/16384/248227184
postgres 25896 postgres  205u   REG      8,1 1073741824 2147488720
/raid/data92/base/16384/248227184.1
postgres 25896 postgres  206u   REG      8,1 1073741824 2147488729
/raid/data92/base/16384/248227184.2
postgres 25896 postgres  207u   REG      8,1  330121216 2147488730
/raid/data92/base/16384/248227184.3
postgres 25896 postgres  208u   REG      8,1 1073741824 2147489600
/raid/data92/base/16384/248227292
postgres 25896 postgres  209u   REG      8,1 1014800384 2147489602
/raid/data92/base/16384/248227292.1
postgres 25896 postgres  210u   REG      8,1 1073741824 2147548643
/raid/data92/base/16384/235421304
postgres 25896 postgres  211u   REG      8,1 1073741824 2147490125
/raid/data92/base/16384/235421304.1
postgres 25896 postgres  212u   REG      8,1 1073741824 2147490308
/raid/data92/base/16384/235421304.2
postgres 25896 postgres  213u   REG      8,1 1073741824 2147503923
/raid/data92/base/16384/235421304.3
postgres 25896 postgres  214u   REG      8,1 1073741824 2147548318
/raid/data92/base/16384/235421304.4
postgres 25896 postgres  215u   REG      8,1 1048870912 2147490124
/raid/data92/base/16384/235421304.5
postgres 25896 postgres  216u   REG      8,1 1073741824 2147616713
/raid/data92/base/16384/247489191
postgres 25896 postgres  217u   REG      8,1 1073741824 2147616715
/raid/data92/base/16384/247489191.1
postgres 25896 postgres  218u   REG      8,1 1073741824 2147616716
/raid/data92/base/16384/247489191.2
postgres 25896 postgres  219u   REG      8,1   36339712 2147616717
/raid/data92/base/16384/247489191.3
postgres 25896 postgres  220u   REG      8,1 1073741824 2147616719
/raid/data92/base/16384/247489565
postgres 25896 postgres  221u   REG      8,1  841646080 2147616722
/raid/data92/base/16384/247489565.1
postgres 25896 postgres  222u   REG      8,1 1073741824 2147548533
/raid/data92/base/16384/240106277
postgres 25896 postgres  223u   REG      8,1 1073741824 2147503894
/raid/data92/base/16384/240106277.1
postgres 25896 postgres  224u   REG      8,1 1073741824 2147489616
/raid/data92/base/16384/240106277.2
postgres 25896 postgres  225u   REG      8,1 1073741824 2147488331
/raid/data92/base/16384/240106277.3
postgres 25896 postgres  226u   REG      8,1 1073741824 2147486555
/raid/data92/base/16384/240106277.4
postgres 25896 postgres  227u   REG      8,1 1073741824 2147490338
/raid/data92/base/16384/240106277.5
postgres 25896 postgres  228u   REG      8,1  564928512 2147486229
/raid/data92/base/16384/240106277.6
postgres 25896 postgres  229u   REG      8,1 1073741824 2147488460
/raid/data92/base/16384/248222581
postgres 25896 postgres  230u   REG      8,1 1073741824 2147488461
/raid/data92/base/16384/248222581.1
postgres 25896 postgres  231u   REG      8,1 1073741824 2147488462
/raid/data92/base/16384/248222581.2
postgres 25896 postgres  232u   REG      8,1  333545472 2147488468
/raid/data92/base/16384/248222581.3
postgres 25896 postgres  233u   REG      8,1 1073741824 2147488535
/raid/data92/base/16384/248222760
postgres 25896 postgres  234u   REG      8,1 1017339904 2147488538
/raid/data92/base/16384/248222760.1
postgres 25896 postgres  235u   REG      8,1 1073741824 2147503943
/raid/data92/base/16384/248123192
postgres 25896 postgres  236u   REG      8,1 1073741824 2147491152
/raid/data92/base/16384/248123192.1
postgres 25896 postgres  237u   REG      8,1 1073741824 2147503957
/raid/data92/base/16384/248123197
postgres 25896 postgres  238u   REG      8,1 1073741824 2147548263
/raid/data92/base/16384/248123198
postgres 25896 postgres  239u   REG      8,1 1073741824 2147486349
/raid/data92/base/16384/241329068
postgres 25896 postgres  240u   REG      8,1 1073741824 2147617614
/raid/data92/base/16384/241329068.1
postgres 25896 postgres  241u   REG      8,1 1073741824 2147617755
/raid/data92/base/16384/241329068.2
postgres 25896 postgres  242u   REG      8,1 1073741824 2147548284
/raid/data92/base/16384/241329068.3
postgres 25896 postgres  243u   REG      8,1 1073741824 2147488470
/raid/data92/base/16384/241329068.4
postgres 25896 postgres  244u   REG      8,1 1073741824 2147490209
/raid/data92/base/16384/241329068.5
postgres 25896 postgres  245u   REG      8,1  541220864 2147499910
/raid/data92/base/16384/241329068.6
postgres 25896 postgres  246u   REG      8,1 1073741824 2147488541
/raid/data92/base/16384/248222987
postgres 25896 postgres  247u   REG      8,1 1073741824 2147488545
/raid/data92/base/16384/248222987.1
postgres 25896 postgres  248u   REG      8,1 1073741824 2147488553
/raid/data92/base/16384/248222987.2
postgres 25896 postgres  249u   REG      8,1  324403200 2147488569
/raid/data92/base/16384/248222987.3
postgres 25896 postgres  250u   REG      8,1 1073741824 2147488587
/raid/data92/base/16384/248223113
postgres 25896 postgres  251u   REG      8,1 1011531776 2147488588
/raid/data92/base/16384/248223113.1
postgres 25896 postgres  252u   REG      8,1 1073741824 2147548655
/raid/data92/base/16384/239039321
postgres 25896 postgres  253u   REG      8,1 1073741824 2147486176
/raid/data92/base/16384/239039321.1
postgres 25896 postgres  254u   REG      8,1 1073741824 2147616667
/raid/data92/base/16384/239039321.2
postgres 25896 postgres  255u   REG      8,1 1073741824 2147747219
/raid/data92/base/16384/239039321.3
postgres 25896 postgres  256u   REG      8,1 1073741824 2147488686
/raid/data92/base/16384/239039321.4
postgres 25896 postgres  257u   REG      8,1 1073741824 2147490049
/raid/data92/base/16384/239039321.5
postgres 25896 postgres  258u   REG      8,1  101154816 2147548526
/raid/data92/base/16384/239039321.6
postgres 25896 postgres  259u   REG      8,1 1073741824 2147486212
/raid/data92/base/16384/248222328
postgres 25896 postgres  260u   REG      8,1 1073741824 2147486221
/raid/data92/base/16384/248222328.1
postgres 25896 postgres  261u   REG      8,1 1073741824 2147486231
/raid/data92/base/16384/248222328.2
postgres 25896 postgres  262u   REG      8,1   96174080 2147486336
/raid/data92/base/16384/248222328.3
postgres 25896 postgres  263u   REG      8,1 1073741824 2147486386
/raid/data92/base/16384/248222469
postgres 25896 postgres  264u   REG      8,1  878927872 2147488459
/raid/data92/base/16384/248222469.1
postgres 25896 postgres  265u   REG      8,1 1073741824 2147503959
/raid/data92/base/16384/237352048
postgres 25896 postgres  266u   REG      8,1 1073741824 2147747466
/raid/data92/base/16384/237352048.1
postgres 25896 postgres  267u   REG      8,1 1073741824 2147486549
/raid/data92/base/16384/237352048.2
postgres 25896 postgres  268u   REG      8,1 1073741824 2147617729
/raid/data92/base/16384/237352048.3
postgres 25896 postgres  269u   REG      8,1 1073741824 2147548639
/raid/data92/base/16384/237352048.4
postgres 25896 postgres  270u   REG      8,1 1073741824 2147488691
/raid/data92/base/16384/237352048.5
postgres 25896 postgres  271u   REG      8,1  268017664 2147503895
/raid/data92/base/16384/237352048.6
postgres 25896 postgres  272u   REG      8,1 1073741824 2147488425
/raid/data92/base/16384/248217976
postgres 25896 postgres  273u   REG      8,1 1073741824 2147488427
/raid/data92/base/16384/248217976.1
postgres 25896 postgres  274u   REG      8,1 1073741824 2147488429
/raid/data92/base/16384/248217976.2
postgres 25896 postgres  275u   REG      8,1  182755328 2147488432
/raid/data92/base/16384/248217976.3
postgres 25896 postgres  276u   REG      8,1 1073741824 2147488433
/raid/data92/base/16384/248218465
postgres 25896 postgres  277u   REG      8,1  927940608 2147488442
/raid/data92/base/16384/248218465.1
postgres 25896 postgres  278u   REG      8,1 1073741824 2147548567
/raid/data92/base/16384/243999874
postgres 25896 postgres  279u   REG      8,1 1073741824 2147499891
/raid/data92/base/16384/243999874.1
postgres 25896 postgres  280u   REG      8,1 1073741824 2147490243
/raid/data92/base/16384/243999874.2
postgres 25896 postgres  281u   REG      8,1 1073741824 2147488163
/raid/data92/base/16384/243999874.3
postgres 25896 postgres  282u   REG      8,1 1073741824 2147616765
/raid/data92/base/16384/243999874.4
postgres 25896 postgres  283u   REG      8,1 1073741824 2147490293
/raid/data92/base/16384/243999874.5
postgres 25896 postgres  284u   REG      8,1  351633408 2147491085
/raid/data92/base/16384/243999874.6
postgres 25896 postgres  285u   REG      8,1 1073741824 2147489647
/raid/data92/base/16384/248231396
postgres 25896 postgres  286u   REG      8,1 1073741824 2147489662
/raid/data92/base/16384/248231396.1
postgres 25896 postgres  287u   REG      8,1 1073741824 2147490053
/raid/data92/base/16384/248231396.2
postgres 25896 postgres  288u   REG      8,1  226844672 2147490054
/raid/data92/base/16384/248231396.3
postgres 25896 postgres  289u   REG      8,1 1073741824 2147490058
/raid/data92/base/16384/248231672
postgres 25896 postgres  290u   REG      8,1  954056704 2147490061
/raid/data92/base/16384/248231672.1
postgres 25896 postgres  291u   REG      8,1 1073741824 2147747623
/raid/data92/base/16384/247248188
postgres 25896 postgres  292u   REG      8,1 1073741824 2147548454
/raid/data92/base/16384/247248188.1
postgres 25896 postgres  293u   REG      8,1 1073741824 2147491162
/raid/data92/base/16384/247248188.2
postgres 25896 postgres  294u   REG      8,1 1073741824 2147488165
/raid/data92/base/16384/247248188.3
postgres 25896 postgres  295u   REG      8,1 1073741824 2147747482
/raid/data92/base/16384/247248188.4
postgres 25896 postgres  296u   REG      8,1 1073741824 2147503918
/raid/data92/base/16384/247248188.5
postgres 25896 postgres  297u   REG      8,1  477880320 2147503941
/raid/data92/base/16384/247248188.6
postgres 25896 postgres  298u   REG      8,1 1073741824 2147747625
/raid/data92/base/16384/247248193
postgres 25896 postgres  299u   REG      8,1 1073741824 2147491149
/raid/data92/base/16384/247248193.1
postgres 25896 postgres  300u   REG      8,1 1073741824 2147486710
/raid/data92/base/16384/247248193.2
postgres 25896 postgres  301u   REG      8,1 1073741824 2147747480
/raid/data92/base/16384/247248193.3
postgres 25896 postgres  302u   REG      8,1 1073741824 2147503922
/raid/data92/base/16384/247248193.4
postgres 25896 postgres  303u   REG      8,1  224124928 2147503942
/raid/data92/base/16384/247248193.5
postgres 25896 postgres  304u   REG      8,1 1073741824 2147747633
/raid/data92/base/16384/247248194
postgres 25896 postgres  305u   REG      8,1 1073741824 2147491438
/raid/data92/base/16384/247248194.1
postgres 25896 postgres  306u   REG      8,1 1073741824 2147747481
/raid/data92/base/16384/247248194.2
postgres 25896 postgres  307u   REG      8,1  360849408 2147490159
/raid/data92/base/16384/247248194.3
postgres 25896 postgres  308u   REG      8,1   72196096 2147486026
/raid/data92/base/16384/12607
postgres 25896 postgres  309u   REG      8,1    3497984 2147486028
/raid/data92/base/16384/12611
postgres 25896 postgres  310u   REG      8,1  206946304 2147486027
/raid/data92/base/16384/12609
postgres 25896 postgres  311u   REG      8,1      16384 2147483822
/raid/data92/global/12754
postgres 25896 postgres  312u   REG      8,1       8192 2147483814
/raid/data92/global/12752
postgres 25896 postgres  313u   REG      8,1      32768 2147483790
/raid/data92/base/16384/12729
postgres 25896 postgres  314u   REG      8,1      16384 2147483829
/raid/data92/base/16384/12720



when I log into bouncer, it does not show 60875 as a client port by
anything. so there was nothing from bouncer to show AFAIK.

10u is the connect glue between bouncer and postgres. I know its odd that I
used TCP between bouncer and postgres even though they are on the same box.






 it's almost like keep alives should have kicked in and cleaned this up.



I run somewhat aggressive keep alives:



postgres=3D# show tcp_keepalives_idle;
 tcp_keepalives_idle
---------------------
 30
(1 row)


postgres=3D# show tcp_keepalives_interval;
 tcp_keepalives_interval
-------------------------
 10
(1 row)

postgres=3D# show tcp_keepalives_count;
 tcp_keepalives_count
----------------------
 6
(1 row)




I had to restart -m i postgres to get postgres useable again.


not sure what other information I can provide, but I think there may be
something up with the changes that went in with 9.2 and bulk copy.


-Mark

В списке pgsql-bugs по дате отправления:

Предыдущее
От: Michael Enke
Дата:
Сообщение: Re: BUG #8307: can reproduce: pg_dump: schema with OID xxxxxx does not exist
Следующее
От: Tom Lane
Дата:
Сообщение: Re: COPY process hung in 9.2.x (9.2.4)