Re: Next step towards 64bit XIDs: Switch to FullTransactionId for PGPROC->xid and XLogRecord->xl_xid

Поиск
Список
Период
Сортировка
От Pavel Borisov
Тема Re: Next step towards 64bit XIDs: Switch to FullTransactionId for PGPROC->xid and XLogRecord->xl_xid
Дата
Msg-id CALT9ZEE4AdqYOcBGm_mj9XOYShF5imScZgEvyfYLyuDtPuB-=w@mail.gmail.com
обсуждение исходный текст
Ответ на Re: Next step towards 64bit XIDs: Switch to FullTransactionId for PGPROC->xid and XLogRecord->xl_xid  (Maxim Orlov <orlovmg@gmail.com>)
Список pgsql-hackers
Hi, Maxim, and Happy New Year!

On Mon, 1 Jan 2024 at 10:15, Maxim Orlov <orlovmg@gmail.com> wrote:
On Fri, 29 Dec 2023 at 16:36, Matthias van de Meent <boekewurm+postgres@gmail.com> wrote:

I don't think this is an actionable change, as this wastes 4 more bytes (or 8 with alignment) in nearly all WAL records that don't use the HEAP/HEAP2/XLOG rmgrs, which would then be up to 10 (if not 14, when 64but-aligned) bytes per record. Unless something like [0] gets committed this will add a significant write overhead to all operations, even if they are not doing anything that needs an XID.

Also, I don't think we need to support transactions that stay alive and change things for longer than 2^31 concurrently created transactions, so we could well add a fxid to each WAL segment header (and checkpoint record?) and calculate the fxid of each record as a relative fxid off of that.

Thank you for your reply.  Yes, this is a good idea.  Actually, this is exactly what I was trying to do at first. 
But in this case, we have to add more locks on TransamVariables->nextXid, and I've abandoned the idea. 
Maybe, I should look in this direction.

On Sun, 31 Dec 2023 at 03:44, Michael Paquier <michael@paquier.xyz> wrote:
And FWIW, echoing with Matthias, making these generic structures
arbitrary larger is a non-starter.  We should try to make them
shorter.

Yeah, obviously, this is patch make WAL bigger.  I'll try to look into the idea of fxid calculation, as mentioned above.
It might in part be a "chicken and the egg" situation.  It is very hard to split overall 64xid patch into smaller pieces
with each part been a meaningful and beneficial for current 32xids Postgres.

The discussion quoted by you as OP [0] shows that there is a wide range of opinions on whether we need 64-bit XIDs and what is the better way to implement it. We can also continue discussing implementation details in that thread [0].

I agree that the step towards something big should not make things worse just now. Does increasing the WAL header for several bytes make performance change detectable at all? Maybe it's worth simulating a workload with a large amount of WAL records and seeing how it works. I think it's regarding this patchset only and could remove or substantiate the main questions about the current patchset.


Kind regards,
Pavel Borisov. 

В списке pgsql-hackers по дате отправления:

Предыдущее
От: Dean Rasheed
Дата:
Сообщение: Re: Adding OLD/NEW support to RETURNING
Следующее
От: Amit Kapila
Дата:
Сообщение: Re: Synchronizing slots from primary to standby