We are trying to speed up real cases, not just benchmarks.
So +1 for the concept, patch is going in right direction though lets do the full press-up.
I have mentioned above the reason for not doing it for sub transactions, if
you think it is viable to reserve space in shared memory for this purpose, then
I can include the optimization for subtransactions as well.
The number of subxids is unbounded, so as you say, reserving shmem isn't viable.
I'm interested in real world cases, so allocating 65 xids per process isn't needed, but we can say is that the optimization shouldn't break down abruptly in the presence of a small/reasonable number of subtransactions.
I think in that case what we can do is if the total number of
sub transactions is lesser than equal to 64 (we can find that by
overflowed flag in PGXact) , then apply this optimisation, else use
the existing flow to update the transaction status. I think for that we
don't even need to reserve any additional memory. Does that sound
sensible to you?
I understand you to mean that the leader should look backwards through the queue collecting xids while !(PGXACT->overflowed)
Yes, that is what the above idea is, but the problem with that is leader
won't be able to collect the subxids of member proc's (from each member
proc's XidCache) as it doesn't have information which of those subxid's
needs to be update as part of current transaction status update (for
subtransactions on different clog pages, we update the status of those
in multiple phases). I think it could only be possible to use the above idea
if all the subtransactions are on same page, which we can identify in
function TransactionIdSetPageStatus(). Though it looks okay that we
can apply this optimization when number of subtransactions is lesser
than 65 and all exist on same page, still it would be better if we can
apply it generically for all cases when number of subtransactions is small
(say 32 or 64). Does this explanation clarify the problem with the above
idea to handle subtransactions?
No additional shmem is required
If we want to do it for all cases when number of subtransactions