Обсуждение: Very frequent "Too many clients" eventually crashes postmaster

Поиск
Список
Период
Сортировка

Very frequent "Too many clients" eventually crashes postmaster

От
"Abraham, Danny"
Дата:
Is there a way to eliminate postmaster crash/core ?

A surge in short-term connections eventually crashes PG on  "no free slots in PMChildFlags array".

Happens on AIX 7.2

=>grep "2020-05-02 08:16" * | grep clients | wc -l
       0
=>grep "2020-05-02 08:17" * | grep clients | wc -l
       0
=>grep "2020-05-02 08:18" * | grep clients | wc -l
     309
=>grep "2020-05-02 08:19" * | grep clients | wc -l
     603
=>grep "2020-05-02 08:20" * | grep clients | wc -l
     288
=>grep "2020-05-02 08:21" * | grep clients | wc -l
     230

2020-05-02 08:21:27.653 CDTFATAL:  no free slots in PMChildFlags array

Re: Very frequent "Too many clients" eventually crashes postmaster

От
Tom Lane
Дата:
"Abraham, Danny" <danny_abraham@bmc.com> writes:
> Is there a way to eliminate postmaster crash/core ?
> A surge in short-term connections eventually crashes PG on  "no free slots in PMChildFlags array".

Update to a current release, maybe?  We fixed a bug with symptoms like
that last fall [1].

            regards, tom lane

[1] https://git.postgresql.org/gitweb/?p=postgresql.git&a=commitdiff&h=3887e9455

Re: Very frequent "Too many clients" eventually crashes postmaster

От
Adrian Klaver
Дата:
On 5/3/20 7:28 AM, Abraham, Danny wrote:
> Is there a way to eliminate postmaster crash/core ?
> 
> A surge in short-term connections eventually crashes PG on  "no free slots in PMChildFlags array".
> 
> Happens on AIX 7.2
> 
> =>grep "2020-05-02 08:16" * | grep clients | wc -l
>         0
> =>grep "2020-05-02 08:17" * | grep clients | wc -l
>         0
> =>grep "2020-05-02 08:18" * | grep clients | wc -l
>       309
> =>grep "2020-05-02 08:19" * | grep clients | wc -l
>       603
> =>grep "2020-05-02 08:20" * | grep clients | wc -l
>       288
> =>grep "2020-05-02 08:21" * | grep clients | wc -l
>       230
> 
> 2020-05-02 08:21:27.653 CDTFATAL:  no free slots in PMChildFlags array
> 

Well:

backend/storage/ipc/pmsignal.c
/* Out of slots ... should never happen, else postmaster.c messed up */
         elog(FATAL, "no free slots in PMChildFlags array");
         return 0;                                       /* keep 
compiler quiet */


So:

1) What are your connection settings?

2) What is generating the client traffic?

3) Is there a pooler in front of the server?



-- 
Adrian Klaver
adrian.klaver@aklaver.com

RE: Re: Very frequent "Too many clients" eventually crashes postmaster

От
"Abraham, Danny"
Дата:
Will setting max_parallel_workers_per_gather  to 0
(In fact, no parallel queries)
Solve this issue?
I can sustain the slowness.
We are on 11.5, and the fix is on 11.6.

-----Original Message-----
From: Tom Lane <tgl@sss.pgh.pa.us>
Sent: Sunday, May 03, 2020 5:37 PM
To: Abraham, Danny <danny_abraham@bmc.com>
Cc: pgsql-general@postgresql.org
Subject: [EXTERNAL] Re: Very frequent "Too many clients" eventually crashes postmaster

"Abraham, Danny" <danny_abraham@bmc.com> writes:
> Is there a way to eliminate postmaster crash/core ?
> A surge in short-term connections eventually crashes PG on  "no free slots in PMChildFlags array".

Update to a current release, maybe?  We fixed a bug with symptoms like that last fall [1].

            regards, tom lane

[1]
https://urldefense.proofpoint.com/v2/url?u=https-3A__git.postgresql.org_gitweb_-3Fp-3Dpostgresql.git-26a-3Dcommitdiff-26h-3D3887e9455&d=DwIFAg&c=UrUhmHsiTVT5qkaA4d_oSzcamb9hmamiCDMzBAEwC7E&r=rZ6QRG5QsrF-veoN0_ehDrT6IsQzka7-fVfEhCfor3I&m=LoZHLvaRhw48Dvx9n1OtgDV_SA0hse7hNTIl0fZh0RQ&s=u3c5XpUBsGhiPxGk5KFwOO_Aj-ToS4YN4pPBuVVZRGI&e=