Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

Поиск
Список
Период
Сортировка
От Ajin Cherian
Тема Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions
Дата
Msg-id CAFPTHDa-LedPs7vNwMyWw_dediE50vPXiN2NDqv32ei=1KKk3A@mail.gmail.com
обсуждение исходный текст
Ответ на Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions  (Amit Kapila <amit.kapila16@gmail.com>)
Ответы Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions  (Amit Kapila <amit.kapila16@gmail.com>)
Список pgsql-hackers
Дерево обсуждения


On Thu, Jul 9, 2020 at 12:28 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
On Wed, Jul 8, 2020 at 7:31 PM Ajin Cherian <itsajin@gmail.com> wrote:

Thanks for showing the interest in patch.  How have you ensured that
streaming is happening?  I don't think the proposed patch can ensure
it for every case because we also rely on logical_decoding_work_mem to
decide whether to stream/spill, see ReorderBufferCheckMemoryLimit.  I
think with your patch it will allow streaming for cases where we have
large amount of WAL to decode.


Maybe I missed something but I looked at ReorderBufferCheckMemoryLimit, even there it checks the same function ReorderBufferCanStream () and decides whether to stream or spill. Did I miss something?

    while (rb->size >= logical_decoding_work_mem * 1024L)
    {
        /*
         * Pick the largest transaction (or subtransaction) and evict it from
         * memory by streaming, if supported. Otherwise, spill to disk.
         */
        if (ReorderBufferCanStream(rb) &&
            (txn = ReorderBufferLargestTopTXN(rb)) != NULL)
        {
            /* we know there has to be one, because the size is not zero */
            Assert(txn && !txn->toptxn);
            Assert(txn->total_size > 0);
            Assert(rb->size >= txn->total_size);

            ReorderBufferStreamTXN(rb, txn);
        }
        else
        { 

I will also add debug and test as you suggested.

regards,
Ajin Cherian
Fujitsu Australia

В списке pgsql-hackers по дате отправления:

Предыдущее
От: Tom Lane
Дата:
Сообщение: Re: Is this a bug in pg_current_logfile() on Windows?
Следующее
От: Amit Kapila
Дата:
Сообщение: Re: Resetting spilled txn statistics in pg_stat_replication