Обсуждение: Optimize SnapBuildPurgeOlderTxn to purge committed xids in-place

Поиск
Список
Период
Сортировка

Optimize SnapBuildPurgeOlderTxn to purge committed xids in-place

От
Neil Chen
Дата:
Hi all,

While reading the code of SnapBuildPurgeOlderTxn, I noticed the TODO comment suggesting a neater algorithm instead of using a temporary workspace for copying committed xids.

I’ve implemented an in-place purge approach for the committed xid array to eliminate the overhead of memory allocation and data copying.
 
Tested locally without issues—happy to hear your thoughts.
Вложения

Re: Optimize SnapBuildPurgeOlderTxn to purge committed xids in-place

От
Xuneng Zhou
Дата:
Hi Neil,

On Thu, Jan 8, 2026 at 11:35 AM Neil Chen <carpenter.nail.cz@gmail.com> wrote:
>
> Hi all,
>
> While reading the code of SnapBuildPurgeOlderTxn, I noticed the TODO comment suggesting a neater algorithm instead of
usinga temporary workspace for copying committed xids. 
>
> I’ve implemented an in-place purge approach for the committed xid array to eliminate the overhead of memory
allocationand data copying. 
>
> Tested locally without issues—happy to hear your thoughts.

Thanks for working on this. I noticed the same issue and proposed a
similar patch before. Here's Michael's feedback on it [1].

You can refer to the comment above the xip array for a discussion of
the trade-offs involved in keeping the array sorted. My later thought
is to maintain the array in sorted order and use a binary search to
identify the range of entries to remove, which should be more
efficient in the common case. That said, this approach still needs
benchmarking, particularly to observe potential worst-case regressions
if it is not implemented in a two-phase manner. In case you might be
interested in it. [2] [3]

* TODO: It's unclear whether that reasoning has much merit. Every
* time we add something here after becoming consistent will also
* require distributing a snapshot. Storing them sorted would
* potentially also make it easier to purge (but more complicated wrt
* wraparound?). Should be improved if sorting while building the
* snapshot shows up in profiles.
*/
TransactionId *xip;

[1] https://www.postgresql.org/message-id/aPbGOWbMrHXgnY6j@paquier.xyz
[2] https://www.postgresql.org/message-id/flat/CABPTF7V9gcpTLrOY0fG4YontoHjVg8YrbmiH4XB_5PT6K56xhg%40mail.gmail.com
[3] https://www.postgresql.org/message-id/flat/CABPTF7XiwbA38OZBj5Y2F-q%2BfZ%3D03YFN9iFnb_406F4SfE-f4w%40mail.gmail.com

--
Best,
Xuneng



Re: Optimize SnapBuildPurgeOlderTxn to purge committed xids in-place

От
Neil Chen
Дата:
Hi Xuneng,

On Thu, Jan 8, 2026 at 12:49 PM Xuneng Zhou <xunengzhou@gmail.com> wrote:

Thanks for working on this. I noticed the same issue and proposed a
similar patch before. Here's Michael's feedback on it [1].

[1] https://www.postgresql.org/message-id/aPbGOWbMrHXgnY6j@paquier.xyz
[2] https://www.postgresql.org/message-id/flat/CABPTF7V9gcpTLrOY0fG4YontoHjVg8YrbmiH4XB_5PT6K56xhg%40mail.gmail.com
[3] https://www.postgresql.org/message-id/flat/CABPTF7XiwbA38OZBj5Y2F-q%2BfZ%3D03YFN9iFnb_406F4SfE-f4w%40mail.gmail.com


Thanks for pointing this out. I will read through the references you provided in detail. 
Let’s continue the discussion in your thread.