Re: POC: postgres_fdw insert batching

Поиск
Список
Период
Сортировка
От Amit Langote
Тема Re: POC: postgres_fdw insert batching
Дата
Msg-id CA+HiwqESrO5BKHtWZZTSbK26OWk-Uhx6DYtdVEkpdaoZ=uW5BA@mail.gmail.com
обсуждение исходный текст
Ответ на RE: POC: postgres_fdw insert batching  ("tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com>)
Ответы RE: POC: postgres_fdw insert batching  ("tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com>)
Список pgsql-hackers
On Sat, Jan 16, 2021 at 12:00 AM tsunakawa.takay@fujitsu.com
<tsunakawa.takay@fujitsu.com> wrote:
> From: Amit Langote <amitlangote09@gmail.com>
> > Okay, so maybe not moving the whole logic into the FDW's
> > BeginForeignModify(), but at least if we move this...
> >
> > @@ -441,6 +449,72 @@ ExecInsert(ModifyTableState *mtstate,
> > +       /*
> > +        * Determine if the FDW supports batch insert and determine the
> > batch
> > +        * size (a FDW may support batching, but it may be disabled for the
> > +        * server/table). Do this only once, at the beginning - we don't want
> > +        * the batch size to change during execution.
> > +        */
> > +       if (resultRelInfo->ri_FdwRoutine->GetForeignModifyBatchSize &&
> > +           resultRelInfo->ri_FdwRoutine->ExecForeignBatchInsert &&
> > +           resultRelInfo->ri_BatchSize == 0)
> > +           resultRelInfo->ri_BatchSize =
> > +
> > resultRelInfo->ri_FdwRoutine->GetForeignModifyBatchSize(resultRelInfo);
> >
> > ...into ExecInitModifyTable(), ExecInsert() only needs the following block:
>
> Does ExecInitModifyTable() know all leaf partitions where the tuples produced by VALUES or SELECT go?  ExecInsert()
doesn'tfind the target leaf partition for the first time through the call to ExecPrepareTupleRouting()?  Leaf
partitionscan have different batch_size settings. 

Good thing you reminded me that this is about inserts, and in that
case no, ExecInitModifyTable() doesn't know all leaf partitions, it
only sees the root table whose batch_size doesn't really matter.  So
it's really ExecInitRoutingInfo() that I would recommend to set
ri_BatchSize; right after this block:

/*
 * If the partition is a foreign table, let the FDW init itself for
 * routing tuples to the partition.
 */
if (partRelInfo->ri_FdwRoutine != NULL &&
    partRelInfo->ri_FdwRoutine->BeginForeignInsert != NULL)
    partRelInfo->ri_FdwRoutine->BeginForeignInsert(mtstate, partRelInfo);

Note that ExecInitRoutingInfo() is called only once for a partition
when it is initialized after being inserted into for the first time.

For a non-partitioned targets, I'd still say set ri_BatchSize in
ExecInitModifyTable().

--
Amit Langote
EDB: http://www.enterprisedb.com



В списке pgsql-hackers по дате отправления:

Предыдущее
От: "tsunakawa.takay@fujitsu.com"
Дата:
Сообщение: RE: POC: postgres_fdw insert batching
Следующее
От: Tomas Vondra
Дата:
Сообщение: Re: list of extended statistics on psql