Maxim, maybe it's still a good idea to isolate those two patches and submit them separately first, to reduce the size of the rest of the patch?
Looks to me like the best next actions would be:
1. Submit a patch that uses XID_FMT everywhere, as a cosmetic change.
This looks like it will reduce the main patch size considerably and
make it much less scary. That can be cleaned up and committed while we
discuss the main approach.
2. Write up the approach in a detailed README, so people can
understand the proposal and assess if there are problems. A few short
notes and a link back to old conversations isn't enough to allow wide
review and give confidence on such a major patch.
Big thanks to all for your ideas!
We intend to do the following work on the patch soon:
1. Write a detailed README
2. Split the patch into several pieces including a separate part for XID_FMT. But if committers meanwhile choose to commit Jim's XID_FMT patch we also appreciate this and will rebase our patch accordingly.
2A. Probably refactor it to store precalculated XMIN/XMAX in memory tuple representation instead of t_xid_base/t_multi_base
2B. Split the in-memory part of a patch as a separate
3. Construct some variants for leaving "double xmax" format as a temporary one just after upgrade for having only one persistent on-disk format instead of two.
3A. By using SQL function "vacuum doublexmax;"
OR
3B. By freeing space on all heap pages for pd_special before pg-upgrade.
OR
3C. By automatically repacking all "double xmax" pages after upgrade (with a priority specified by common vacuum-related GUCs)
4. Intentionally prohibit starting a new transaction with XID difference of more than 2^32 from the oldest currently running one. This is to enforce some dba's action for cleaning defunct transaction but not binding one: he/she can wait if they consider these old transactions not defunct.
5. Investigate and add a solution for archs without 64-bit atomic values.
5A. Provide XID 8-byte alignment for systems where 64-bit atomics is provided for 8-byte aligned values.
5B. Wrap XID reading into PG atomic locks for remaining 32-bit ones (they are expected to be rare).
--