On 2/9/21 12:47 PM, tsunakawa.takay@fujitsu.com wrote:
> From: Andrey V. Lepikhov <a.lepikhov@postgrespro.ru>
> I guess you used many hash partitions. Sadly, The current COPY implementation only accumulates either 1,000 rows or
64KB of input data (very small!) before flushing all CopyMultiInsertBuffers. One CopyMultiInsertBuffer corresponds to
onepartition. Flushing a CopyMultiInsertBuffer calls ExecForeignCopy() once, which connects to a remote database, runs
COPYFROM STDIN, and disconnects. Here, the flushing trigger (1,000 rows or 64 KB input data, whichever comes first) is
sosmall that if there are many target partitions, the amount of data for each partition is small.
I tried to use 1E4 - 1E8 rows in a tuple buffer. But the results weren't
impressive.
We can use one more GUC instead of a precompiled constant.
> Why don't we focus on committing the basic part and addressing the extended part (0003 and 0004) separately later?
I focused only on the 0001 and 0002 patches.
> As Tang-san and you showed, the basic part already demonstrated impressive improvement. If there's no objection,
I'dlike to make this ready for committer in a few days.
Good.
--
regards,
Andrey Lepikhov
Postgres Professional