Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

Hi,

Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>, 6 Tem 2023 Per,
12:47 tarihinde şunu yazdı:
>
> Dear Melih,
>
> > Thanks for the 0003 patch. But it did not work for me. Can you create
> > a subscription successfully with patch 0003 applied?
> > I get the following error: " ERROR:  table copy could not start
> > transaction on publisher: another command is already in progress".
>
> You got the ERROR when all the patches (0001-0005) were applied, right?
> I have focused on 0001 and 0002 only, so I missed something.
> If it was not correct, please attach the logfile and test script what you did.

Yes, I did get an error with all patches applied. But with only 0001
and 0002, your version seems like working and mine does not.
What do you think about combining 0002 and 0003? Or should those stay separate?

> Hi, I did a performance testing for v16 patch set.
> Results show that patches significantly improves the performance in most cases.
>
> # Method
>
> Following tests were done 10 times per condition, and compared by median.
> do_one_test.sh was used for the testing.
>
> 1.      Create tables on publisher
> 2.      Insert initial data on publisher
> 3.      Create tables on subscriber
> 4.      Create a replication slot (mysub_slot) on publisher
> 5.      Create a publication on publisher
> 6.      Create tables on subscriber
> --- timer on ---
> 7.      Create subscription with pre-existing replication slot (mysub_slot)
> 8.      Wait until all srsubstate in pg_subscription_rel becomes 'r'
> --- timer off ---
>

Thanks for taking the time to do testing and sharing the results. This
is also how I've been doing the testing since, but the process was
half scripted, half manual work.

> According to the measurement, we can say following things:
>
> * In any cases the performance was improved from the HEAD.
> * The improvement became more significantly if number of synced tables were increased.

Yes, I believe it becomes more significant when workers spend less
time with actually copying data but more with other stuff like
launching workers, opening connections etc.

> * 0003 basically improved performance from first two patches

Agree, 0003 is definitely a good addition which was missing earlier.


Thanks,
--
Melih Mutlu
Microsoft