bulk_multi_insert infinite loops with large rows and small fill factors

Поиск
Список
Период
Сортировка
От David Gould
Тема bulk_multi_insert infinite loops with large rows and small fill factors
Дата
Msg-id 20121212030419.71b1e974@jekyl.davidgould.org
обсуждение исходный текст
Ответы Re: bulk_multi_insert infinite loops with large rows and small fill factors
Список pgsql-hackers
COPY IN loops in heap_multi_insert() extending the table until it fills the
disk when trying to insert a wide row into a table with a low fill-factor.
Internally fill-factor is implemented by reserving some space space on a
page. For large enough rows and small enough fill-factor bulk_multi_insert()
can't fit the row even on a new empty page, so it keeps allocating new pages
but is never able to place the row. It should always put at least one row on
an empty page.

In the excerpt below saveFreeSpace is the reserved space for the fill-factor.

    while (ndone < ntuples)
    { ...
        /*
         * Find buffer where at least the next tuple will fit.  If the page is
         * all-visible, this will also pin the requisite visibility map page.
         */
        buffer = RelationGetBufferForTuple(relation, heaptuples[ndone]->t_len,
        ...
        /* Put as many tuples as fit on this page */
        for (nthispage = 0; ndone + nthispage < ntuples; nthispage++)
        {
            HeapTuple   heaptup = heaptuples[ndone + nthispage];

            if (PageGetHeapFreeSpace(page) < MAXALIGN(heaptup->t_len) + saveFreeSpace)
                break;

            RelationPutHeapTuple(relation, buffer, heaptup);
        }
        ...Do a bunch of dirtying and logging etc ...
     }

This was introduced in 9.2 as part of the bulk insert speedup.

One more point, in the case where we don't insert any rows, we still do all the
dirtying and logging work even though we did not modify the page. I have tried
skip all this if no rows are added (nthispage == 0), but my access method foo
is sadly out of date, so someone should take a skeptical look at that.

A test case and patch against 9.2.2 is attached. It fixes the problem and passes
make check. Most of the diff is just indentation changes. Whoever tries this will
want to test this on a small partition by itself.

-dg

--
David Gould              510 282 0869         daveg@sonic.net
If simplicity worked, the world would be overrun with insects.

Вложения

В списке pgsql-hackers по дате отправления:

Предыдущее
От: Andres Freund
Дата:
Сообщение: Re: Logical decoding & exported base snapshot
Следующее
От: Andres Freund
Дата:
Сообщение: Re: Logical decoding & exported base snapshot