Re: PG18 GIN parallel index build crash - invalid memory alloc request size
| От | Gregory Smith |
|---|---|
| Тема | Re: PG18 GIN parallel index build crash - invalid memory alloc request size |
| Дата | |
| Msg-id | CAHLJuCWFF=+DxSVQMAeUhgHg=pBP18L3CXyHWOdGdzjZ7E0p7A@mail.gmail.com обсуждение исходный текст |
| Ответ на | Re: PG18 GIN parallel index build crash - invalid memory alloc request size (Tomas Vondra <tomas@vondra.me>) |
| Ответы |
Re: PG18 GIN parallel index build crash - invalid memory alloc request size
|
| Список | pgsql-hackers |
On Fri, Oct 24, 2025 at 8:38 AM Tomas Vondra <tomas@vondra.me> wrote:
Hmmm, I wonder if the m_w_m is high enough to confuse the trimming logic
in some way. Can you try if using smaller m_w_m (maybe 128MB-256MB)
makes the issue go away?
The index builds at up to 4GB of m_w_m. 5GB and above crashes.
Now that I know roughly where the limits are that de-escalates things a bit. The sort of customers deploying a month after release should be OK with just knowing to be careful about high m_w_m settings on PG18 until a fix is ready.
Hope everyone is enjoying Latvia. My obscure music collection includes a band from there I used to see in the NYC area, The Quags; https://www.youtube.com/watch?v=Bg3P4736CxM
Can you show the contents of buffer and tup? I'm especially interested
in these fields:
buffer->nitems
buffer->maxitems
buffer->nfrozen
tup->nitems
I'll see if I can grab that data at the crash point.
FYI for anyone who wants to replicate this: if you have a system with 128GB+ of RAM you could probably recreate the test case. Just have to download the Planet file and run osm2pgsql with the overly tweaked settings I use. I've published all the details of how I run this regression test now.
If I'm right, I think there are two ways to fix this:
(1) apply the trimming earlier, i.e. try to freeze + flush before
actually merging the data (essentially, update nfrozen earlier)
(2) use repalloc_huge (and palloc_huge) in GinBufferStoreTuple
Or we probably should do both.
Sounds like (2) is probably mandatory and (1) is good hygiene.
В списке pgsql-hackers по дате отправления: