Re: BUG #17254: Crash with 0xC0000409 in pg_stat_statements when pg_stat_tmp\pgss_query_texts.stat exceeded 2GB.

Поиск
Список
Период
Сортировка
От Juan José Santamaría Flecha
Тема Re: BUG #17254: Crash with 0xC0000409 in pg_stat_statements when pg_stat_tmp\pgss_query_texts.stat exceeded 2GB.
Дата
Msg-id CAC+AXB2vCUOj6x8drBVUVCgjyy4ZFUu-TFBuNVW=7Emh44EC=w@mail.gmail.com
обсуждение исходный текст
Ответ на Re: BUG #17254: Crash with 0xC0000409 in pg_stat_statements when pg_stat_tmp\pgss_query_texts.stat exceeded 2GB.  (Tom Lane <tgl@sss.pgh.pa.us>)
Ответы Re: BUG #17254: Crash with 0xC0000409 in pg_stat_statements when pg_stat_tmp\pgss_query_texts.stat exceeded 2GB.  (Tom Lane <tgl@sss.pgh.pa.us>)
Список pgsql-bugs

On Sat, Oct 30, 2021 at 6:26 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:

That seems like a pretty poor solution.  It will cause pg_stat_statements
to fail altogether as soon as the stats file exceeds 1GB.  (Admittedly,
failing is better than crashing, but not by that much.)  Worse, it causes
that to happen on EVERY platform, not only Windows where the problem is.

I don't think it is a Windows only problem, even on POSIX platforms it might not be safe trying to read() over 2GB.
 
I think instead, we need to turn the subsequent one-off read() call into a
loop that reads no more than INT_MAX bytes at a time.  It'd be possible
to restrict that to Windows, but probably no harm in doing it the same
way everywhere.

Seems reasonable to me, can such a change be back-patched?
 
A different line of thought is that maybe we shouldn't be letting the
file get so big in the first place.  Letting every backend have its
own copy of a multi-gigabyte stats file is going to be problematic,
and not only on Windows.  It looks like the existing logic just considers
the number of hash table entries, not their size ... should we rearrange
things to keep a running count of the space used?

+1. There should be a mechanism to limit the effective memory size.

Regards,

Juan José Santamaría Flecha

В списке pgsql-bugs по дате отправления:

Предыдущее
От: Tom Lane
Дата:
Сообщение: Re: BUG #17254: Crash with 0xC0000409 in pg_stat_statements when pg_stat_tmp\pgss_query_texts.stat exceeded 2GB.
Следующее
От: Peter Geoghegan
Дата:
Сообщение: Re: BUG #17245: Index corruption involving deduplicated entries