Re: [SPAM] psql 9.3 automatic recovery in progress

On 10/10/2016 12:18 PM, Periko Support wrote:
> I was on vacation, but the issue have the same behavior:

Actually no. Before you had:

2016-09-12 09:00:01 PDT LOG:  server process (PID 23958) was
terminated by signal 9: Killed

Now you have:

2016-10-10 07:50:09 PDT WARNING:  terminating connection because of
crash of another server process
2016-10-10 07:50:09 PDT DETAIL:  The postmaster has commanded this
server process to roll back the current transaction and exit, because
another server process exited abnormally and possibly corrupted shared
memory.

Which corresponds to this from your subsequent post:

#            os.system("kill -9 %s" % (int(pid[0]), ))
             os.kill(int(pid[0]), signal.SIGKILL)

>
> 2016-10-10 07:50:09 PDT WARNING:  terminating connection because of
> crash of another server process
> 2016-10-10 07:50:09 PDT DETAIL:  The postmaster has commanded this
> server process to roll back the current transaction and exit, because
> another server process exited abnormally and possibly corrupted shared
> memory.
> 2016-10-10 07:50:09 PDT HINT:  In a moment you should be able to
> reconnect to the database and repeat your command.
> 2016-10-10 07:50:09 PDT WARNING:  terminating connection because of
> crash of another server process
> 2016-10-10 07:50:09 PDT DETAIL:  The postmaster has commanded this
> server process to roll back the current transaction and exit, because
> another server process exited abnormally and possibly corrupted shared
> memory.
> 2016-10-10 07:50:09 PDT HINT:  In a moment you should be able to
> reconnect to the database and repeat your command.
> 2016-10-10 07:50:09 PDT LOG:  archiver process (PID 13066) exited with
> exit code 1
> 2016-10-10 07:50:10 PDT LOG:  all server processes terminated; reinitializing
> 2016-10-10 07:50:10 PDT LOG:  connection received: host=192.168.2.6 port=37700
> 2016-10-10 07:50:10 PDT LOG:  database system was interrupted; last
> known up at 2016-10-10 07:49:15 PDT
> 2016-10-10 07:50:10 PDT FATAL:  the database system is in recovery mode
> 2016-10-10 07:50:10 PDT LOG:  connection received: host=192.168.2.6 port=37702
> 2016-10-10 07:50:10 PDT FATAL:  the database system is in recovery mode
> 2016-10-10 07:50:15 PDT LOG:  database system was not properly shut
> down; automatic recovery in progress
> 2016-10-10 07:50:15 PDT LOG:  redo starts at 517/C9000028
> 2016-10-10 07:50:15 PDT LOG:  unexpected pageaddr 517/77000000 in log
> segment 0000000100000517000000D2, offset 0
> 2016-10-10 07:50:15 PDT LOG:  redo done at 517/D10000C8
> 2016-10-10 07:50:15 PDT LOG:  last completed transaction was at log
> time 2016-10-10 07:49:09.891669-07
> 2016-10-10 07:50:15 PDT LOG:  connection received: host=192.168.2.6 port=37704
> 2016-10-10 07:50:15 PDT FATAL:  the database system is in recovery mode
> 2016-10-10 07:50:15 PDT LOG:  connection received: host=192.168.2.6 port=37706
> 2016-10-10 07:50:15 PDT FATAL:  the database system is in recovery mode
> 2016-10-10 07:50:16 PDT LOG:  MultiXact member wraparound protections
> are now enabled
> 2016-10-10 07:50:16 PDT LOG:  database system is ready to accept connections
> 2016-10-10 07:50:16 PDT LOG:  autovacuum launcher started
>
> 2016-10-10 09:00:01 PDT LOG:  archiver process (PID 14004) exited with
> exit code 1
> 2016-10-10 09:00:01 PDT WARNING:  terminating connection because of
> crash of another server process
> 2016-10-10 09:00:01 PDT DETAIL:  The postmaster has commanded this
> server process to roll back the current transaction and exit, because
> another server process exited abnormally and possibly corrupted shared
> memory.
> 2016-10-10 09:00:01 PDT HINT:  In a moment you should be able to
> reconnect to the database and repeat your command.
> 2016-10-10 09:00:01 PDT WARNING:  terminating connection because of
> crash of another server process
> 2016-10-10 09:00:01 PDT DETAIL:  The postmaster has commanded this
> server process to roll back the current transaction and exit, because
> another server process exited abnormally and possibly corrupted shared
> memory.
> 2016-10-10 09:00:01 PDT HINT:  In a moment you should be able to
> reconnect to the database and repeat your command.
> 2016-10-10 09:00:01 PDT LOG:  all server processes terminated; reinitializing
> 2016-10-10 09:00:02 PDT LOG:  database system was interrupted; last
> known up at 2016-10-10 08:59:33 PDT
> 2016-10-10 09:00:03 PDT LOG:  connection received: host=127.0.0.1 port=35950
> 2016-10-10 09:00:03 PDT FATAL:  the database system is in recovery mode
> 2016-10-10 09:00:03 PDT LOG:  connection received: host=127.0.0.1 port=35951
> 2016-10-10 09:00:03 PDT FATAL:  the database system is in recovery mode
> 2016-10-10 09:00:03 PDT LOG:  connection received: host=127.0.0.1 port=35952
> 2016-10-10 09:00:03 PDT FATAL:  the database system is in recovery mode
> 2016-10-10 09:00:03 PDT LOG:  connection received: host=127.0.0.1 port=35953
> 2016-10-10 09:00:03 PDT FATAL:  the database system is in recovery mode
> 2016-10-10 09:00:03 PDT LOG:  connection received: host=127.0.0.1 port=35954
> 2016-10-10 09:00:03 PDT FATAL:  the database system is in recovery mode
> 2016-10-10 09:00:03 PDT LOG:  connection received: host=127.0.0.1 port=35955
> 2016-10-10 09:00:03 PDT FATAL:  the database system is in recovery mode
> 2016-10-10 09:00:05 PDT LOG:  connection received: host=192.168.2.6 port=39380
> 2016-10-10 09:00:05 PDT FATAL:  the database system is in recovery mode
> 2016-10-10 09:00:05 PDT LOG:  connection received: host=192.168.2.6 port=39382
> 2016-10-10 09:00:05 PDT FATAL:  the database system is in recovery mode
> 2016-10-10 09:00:07 PDT LOG:  database system was not properly shut
> down; automatic recovery in progress
> 2016-10-10 09:00:07 PDT LOG:  redo starts at 51A/82000028
> 2016-10-10 09:00:07 PDT LOG:  record with zero length at 51A/8E0126B0
> 2016-10-10 09:00:07 PDT LOG:  redo done at 51A/8E012680
> 2016-10-10 09:00:07 PDT LOG:  last completed transaction was at log
> time 2016-10-10 09:00:01.142505-07
> 2016-10-10 09:00:08 PDT LOG:  connection received: host=127.0.0.1 port=35956
> 2016-10-10 09:00:08 PDT FATAL:  the database system is in recovery mode
> 2016-10-10 09:00:08 PDT LOG:  connection received: host=127.0.0.1 port=35957
> 2016-10-10 09:00:08 PDT FATAL:  the database system is in recovery mode
> 2016-10-10 09:00:08 PDT LOG:  MultiXact member wraparound protections
> are now enabled
> 2016-10-10 09:00:08 PDT LOG:  database system is ready to accept connections
> 2016-10-10 09:00:08 PDT LOG:  autovacuum launcher started
>
> Thanks.
>
> On Mon, Oct 10, 2016 at 11:32 AM, Adrian Klaver
> <adrian.klaver@aklaver.com> wrote:
>> On 10/10/2016 11:14 AM, Moreno Andreo wrote:
>>>
>>>
>>> Il 10/10/2016 18:24, Periko Support ha scritto:
>>>>
>>>> 2016-09-12 09:00:01 PDT LOG:  server process (PID 23958) was
>>>> terminated by signal 9: Killed
>>>
>>>
>>>> 2016-09-12 10:00:01 PDT LOG:  server process (PID 30766) was
>>>> terminated by signal 9: Killed
>>>
>>>
>>>> 2016-09-12 15:00:01 PDT LOG:  server process (PID 22030) was
>>>> terminated by signal 9: Killed
>>>>
>>>>
>>> These datetimes could be suspect. Every crash (kill) is done at
>>> "00"minutes and "01" minutes, that makes me ask "Isn't there something
>>> like cron running something that interfere with postgres?"
>>
>>
>> While we on the subject, the datetimes are almost a month old.
>>
>> Does that mean this problem was just noticed or are the datetimes wrong?
>>
>>>
>>> Cheers,
>>> Moreno.
>>>
>>>
>>>
>>>
>>
>>
>> --
>> Adrian Klaver
>> adrian.klaver@aklaver.com
>>
>>
>>
>> --
>> Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
>> To make changes to your subscription:
>> http://www.postgresql.org/mailpref/pgsql-general
>
>


--
Adrian Klaver
adrian.klaver@aklaver.com