Re: Logical replication prefetch

On Sun, Jul 13, 2025 at 5:59 PM Konstantin Knizhnik <knizhnik@garret.ru> wrote:
>
>
> Certainly originally intended use case was different: parallel apply is
> performed only for large transactions. Number of of such transactions is
> not so big and
> so there should be enough parallel apply workers in pool to proceed
> them. And if there are not enough workers, it is not a problem to spawn
> new one and terminate
> it after completion of transaction (because transaction is long,
> overhead of spawning process is not so larger comparing with redo of
> large transaction).
>

Right.

> But if we want to efficiently replicate OLTP workload, then we
> definitely need some other approach.
>

Agreed, for simplicity, for now we can have a GUC to decide the size
of the pool. There is a note in the code for this as well, see: " XXX
This worker pool threshold is arbitrary and we can provide a GUC
variable for this in the future if required. I think we can think of
some dynamic strategy where we remove from the pool if the workers are
not in use for some threshold period of time or something on those
lines. But at this stage it is better to use something simple and try
to come up with a good way to perform pre-fetch or parallelization of
short transactions.

--
With Regards,
Amit Kapila.