Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding

On Fri, Feb 21, 2025 at 12:57 PM Ajin Cherian <itsajin@gmail.com> wrote:
>
> On Thu, Feb 20, 2025 at 3:08 PM Hayato Kuroda (Fujitsu)
> <kuroda.hayato@fujitsu.com> wrote:
> >
> > Dear Ajin,
> >
> > > I compared the patch 1 which does not employ a hash cache and has the
> > > overhead of starting a transaction every time the filter is checked.
> > >
> > > I created a test setup of 10 million inserts in 3 different scenarios:
> > > 1. All inserts on unpublished tables
> > > 2. Half of the inserts on unpublished table and half on pupblished table
> > > 3. All inserts on published tables.
> > >
> > > The percentage improvement in the new optimized patch compared to the
> > > old patch is:
> > >
> > > No transactions in publication: 85.39% improvement
> > > Half transactions in publication: 72.70% improvement
> > > All transactions in publication: 48.47% improvement
> > >
> > > Attaching a graph to show the difference.
> >
> > I could not find any comparisons with HEAD. Can you clarify the throughput/latency/memory
> > usage with HEAD?
>
> Here's the difference in latency with head. Again 10 million inserts
> in 3 scenarios: All transactions on unpublished tables, half of the
> transactions on unpublished tables and all transactions on published
> tables
> Conclusion:
> The patched code with 100 transaction throttling significantly
> improves performance, reducing execution time by ~69% when no
> published transactions are involved, ~43% with partial published
> transactions, and ~15% in all published transactions.
> Attaching a graph showing the performance differences.
>

In these tests, I also see an increased performance with the patch
even when all transactions are published. I will investigate why this
happens and update.

regards,
Ajin Cherian
Fujitsu Australia