Обсуждение: [BUGS] BUG #14835: Postgres crashed
The following bug has been logged on the website: Bug reference: 14835 Logged by: Nicola Contu Email address: nicola.contu@gmail.com PostgreSQL version: 9.6.5 Operating system: Centos 6.9 Description: Hello, after upgrading to postgres 9.6.5 in the last 3weeks we had the db crashing three times for the same reason : SELECT: double free or corruption (!prev): 0x0000000001f4b440 *** ======= Backtrace: ========= /lib64/libc.so.6[0x3eee675dee] /lib64/libc.so.6[0x3eee678c80] ======= Memory map: ======== 00400000-009ff000 r-xp 00000000 fd:00 6431324 /usr/local/pgsql9.6.5/bin/postgres 00bfe000-00c0b000 rw-p 005fe000 fd:00 6431324 /usr/local/pgsql9.6.5/bin/postgres 00c0b000-00c5c000 rw-p 00000000 00:00 0 01dd5000-01e17000 rw-p 00000000 00:00 0 01e17000-0abf7000 rw-p 00000000 00:00 0 3441000000-3441016000 r-xp 00000000 fd:00 7602384 /lib64/libgcc_s-4.4.7-20120601.so.1 3441016000-3441215000 ---p 00016000 fd:00 7602384 /lib64/libgcc_s-4.4.7-20120601.so.1 3441215000-3441216000 rw-p 00015000 fd:00 7602384 /lib64/libgcc_s-4.4.7-20120601.so.1 3eee200000-3eee220000 r-xp 00000000 fd:00 7602252 /lib64/ld-2.12.so (deleted) 3eee41f000-3eee421000 r--p 0001f000 fd:00 7602252 /lib64/ld-2.12.so (deleted) 3eee421000-3eee422000 rw-p 00021000 fd:00 7602252 /lib64/ld-2.12.so (deleted) 3eee422000-3eee423000 rw-p 00000000 00:00 0 3eee600000-3eee78a000 r-xp 00000000 fd:00 7602254 /lib64/libc-2.12.so (deleted) 3eee78a000-3eee98a000 ---p 0018a000 fd:00 7602254 /lib64/libc-2.12.so (deleted) 3eee98a000-3eee98e000 r--p 0018a000 fd:00 7602254 /lib64/libc-2.12.so (deleted) 3eee98e000-3eee990000 rw-p 0018e000 fd:00 7602254 /lib64/libc-2.12.so (deleted) 3eee990000-3eee994000 rw-p 00000000 00:00 0 3eeea00000-3eeea02000 r-xp 00000000 fd:00 7602600 /lib64/libdl-2.12.so (deleted) 3eeea02000-3eeec02000 ---p 00002000 fd:00 7602600 /lib64/libdl-2.12.so (deleted) 3eeec02000-3eeec03000 r--p 00002000 fd:00 7602600 /lib64/libdl-2.12.so (deleted) 3eeec03000-3eeec04000 rw-p 00003000 fd:00 7602600 /lib64/libdl-2.12.so (deleted) 3eeee00000-3eeee17000 r-xp 00000000 fd:00 7602273 /lib64/libpthread-2.12.so (deleted) 3eeee17000-3eef017000 ---p 00017000 fd:00 7602273 /lib64/libpthread-2.12.so (deleted) 3eef017000-3eef018000 r--p 00017000 fd:00 7602273 /lib64/libpthread-2.12.so (deleted) 3eef018000-3eef019000 rw-p 00018000 fd:00 7602273 /lib64/libpthread-2.12.so (deleted) 3eef019000-3eef01d000 rw-p 00000000 00:00 0 3eef600000-3eef683000 r-xp 00000000 fd:00 7602307 /lib64/libm-2.12.so (deleted) 3eef683000-3eef882000 ---p 00083000 fd:00 7602307 /lib64/libm-2.12.so (deleted) 3eef882000-3eef883000 r--p 00082000 fd:00 7602307 /lib64/libm-2.12.so (deleted) 3eef883000-3eef884000 rw-p 00083000 fd:00 7602307 /lib64/libm-2.12.so (deleted) 3eefa00000-3eefa07000 r-xp 00000000 fd:00 7602317 /lib64/librt-2.12.so (deleted) 3eefa07000-3eefc06000 ---p 00007000 fd:00 7602317 /lib64/librt-2.12.so (deleted) 3eefc06000-3eefc07000 r--p 00006000 fd:00 7602317 /lib64/librt-2.12.so (deleted) 3eefc07000-3eefc08000 rw-p 00007000 fd:00 7602317 /lib64/librt-2.12.so (deleted) 7fb338000000-7fb338021000 rw-p 00000000 00:00 0 7fb338021000-7fb33c000000 ---p 00000000 00:00 0 7fb33fe6b000-7fb340b2d000 rw-p 00000000 00:00 0 7fb345603000-7fb345609000 r-xp 00000000 fd:00 5771504 /usr/local/pgsql9.6.5/lib/fuzzystrmatch.so 7fb345609000-7fb345808000 ---p 00006000 fd:00 5771504 /usr/local/pgsql9.6.5/lib/fuzzystrmatch.so 7fb345808000-7fb345809000 rw-p 00005000 fd:00 5771504 /usr/local/pgsql9.6.5/lib/fuzzystrmatch.so 7fb345809000-7fb34688d000 rw-p 00000000 00:00 0 7fb34708e000-7fb347577000 rw-p 00000000 00:00 0 7fb347577000-7fb347584000 r-xp 00000000 fd:00 7602223 /lib64/libnss_files-2.12.so (deleted) 7fb347584000-7fb347783000 ---p 0000d000 fd:00 7602223 /lib64/libnss_files-2.12.so (deleted) 7fb347783000-7fb347784000 r--p 0000c000 fd:00 7602223 /lib64/libnss_files-2.12.so (deleted) 7fb347784000-7fb347785000 rw-p 0000d000 fd:00 7602223 /lib64/libnss_files-2.12.so (deleted) 7fb347794000-7fb5555e0000 rw-s 00000000 00:04 3756858110 /dev/zero (deleted) 7fb5555e0000-7fb55b475000 r--p 00000000 fd:00 5656397 /usr/lib/locale/locale-archive (deleted) 7fb55b475000-7fb55b47a000 rw-p 00000000 00:00 0 7fb55b47e000-7fb55b47f000 rw-p 00000000 00:00 0 7fb55b47f000-7fb55b486000 r--s 00000000 fd:00 5651539 /usr/lib64/gconv/gconv-modules.cache 7fb55b486000-7fb55b488000 rw-s 00000000 00:10 3756858114 /dev/shm/PostgreSQL.353302406 7fb55b488000-7fb55b489000 rw-s 00000000 00:04 87130112 /SYSV0052e2c1 (deleted) 7fb55b489000-7fb55b48a000 rw-p 00000000 00:00 0 7ffd1618d000-7ffd161a2000 rw-p 00000000 00:00 0 [stack] 7ffd161e4000-7ffd161e5000 r-xp 00000000 00:00 0 [vdso] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] Is this a bug? Can you help us on this? Thank you -- Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-bugs
On 09/28/2017 05:46 PM, nicola.contu@gmail.com wrote: > The following bug has been logged on the website: > > Bug reference: 14835 > Logged by: Nicola Contu > Email address: nicola.contu@gmail.com > PostgreSQL version: 9.6.5 > Operating system: Centos 6.9 > Description: > > Hello, > after upgrading to postgres 9.6.5 in the last 3weeks we had the db crashing > three times for the same reason : > > SELECT: double free or corruption (!prev): 0x0000000001f4b440 *** > ======= Backtrace: ========= > /lib64/libc.so.6[0x3eee675dee] > /lib64/libc.so.6[0x3eee678c80] > > > ======= Memory map: ======== > 00400000-009ff000 r-xp 00000000 fd:00 6431324 > /usr/local/pgsql9.6.5/bin/postgres Yes, this seems like a bug, but it's quite unclear if it's in PostgreSQL or due to a bug in some other library. Please, make sure next time a core dump is generated, so that you can inspect it, and generate backtrace for example. You'll probably have to do this: 1) install debuginfo packages (likely using debuginfo-install) 2) enable core files (you'll probably have to set limits in limits.conf and set core_pattern), see [1] for details It might also be useful to know what query was being executed at the time of the crash - that should be logged into the PostgreSQL log. [1] http://man7.org/linux/man-pages/man5/core.5.html regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-bugs
Thanks Tomas.
The query is a select and it takes 4-5s to run. This does not seem the problem.
They were queries that were able to run for a long time (years) and postgres never crashed since the last two updates.
For now there were a few updates on Centos 6.9 (Kernel i.e) so let's see if this will solve the problem.
I'll keep you posted.
Nicola
2017-09-28 19:14 GMT+02:00 Tomas Vondra <tomas.vondra@2ndquadrant.com>:
On 09/28/2017 05:46 PM, nicola.contu@gmail.com wrote:
> The following bug has been logged on the website:
>
> Bug reference: 14835
> Logged by: Nicola Contu
> Email address: nicola.contu@gmail.com
> PostgreSQL version: 9.6.5
> Operating system: Centos 6.9
> Description:
>
> Hello,
> after upgrading to postgres 9.6.5 in the last 3weeks we had the db crashing
> three times for the same reason :
>
> SELECT: double free or corruption (!prev): 0x0000000001f4b440 ***
> ======= Backtrace: =========
> /lib64/libc.so.6[0x3eee675dee]
> /lib64/libc.so.6[0x3eee678c80]
>
>
> ======= Memory map: ========
> 00400000-009ff000 r-xp 00000000 fd:00 6431324
> /usr/local/pgsql9.6.5/bin/postgres
Yes, this seems like a bug, but it's quite unclear if it's in PostgreSQL
or due to a bug in some other library.
Please, make sure next time a core dump is generated, so that you can
inspect it, and generate backtrace for example. You'll probably have to
do this:
1) install debuginfo packages (likely using debuginfo-install)
2) enable core files (you'll probably have to set limits in limits.conf
and set core_pattern), see [1] for details
It might also be useful to know what query was being executed at the
time of the crash - that should be logged into the PostgreSQL log.
[1] http://man7.org/linux/man-pages/man5/core.5.html
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services