On Tue, May 5, 2015 at 8:36 AM, Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
> Thomas Munro wrote:
>
>> FWIW, in some future release, I think we should consider getting a
>> bigger multixact member address space that wraps around at 2^48 or
>> 2^64 instead of 2^32, so that we can sidestep the whole business and
>> go back to having just xid and mxid wraparounds to worry about.
>> pg_multixact/offsets would be 50% or 100% bigger (an extra byte or two
>> per multixact), but it's not very big. pg_multiact/members would be
>> no bigger for any workload that currently works without hitting the
>> wraparound error, but could grow bigger if needed.
>
> Not sure that enlarging the addressable area to 48/64 bits is feasible,
> TBH. We already have many complaints that multixacts take too much disk
> space; we don't want to make that 2^32 times worse, not even 2^16 times
> worse. I don't understand why you say it'd become 1 byte bigger per
> multixact; it would have to be 4 more bytes (2^64) or 2 more bytes
> (2^48), no? If you have 150 million multixacts (the default freeze
> table age) that would mean about 300 or 600 MB of additional disk space,
> which is not insignificant: with the current system, in an database with
> normal multixact usage of 4 members per multixact, members/ would use
> about 2.8 GB, so 600 additional MB in offsets/ is large enough growth to
> raise some more complaints.
Right, sorry, I must have been thinking of 40 bit or 48 bit indexes
when I said 1 or 2 bytes.
I can't help thinking there must be a different way to do this that
takes advantage of the fact that multixacts are often created by
copying all the members of an existing multixact and adding one new
one, so that there is a lot of duplication and churn (at least when
you have a workload that generates bigger multixacts, due to the
O(n^2) process of building them up xid by xid).
Maybe there is a way to store a pointer to some other multixact + a
new xid in a chain structure, but I don't see how to do the cleanup
when you have active multixacts with backwards references to older
multixacts.
Maybe you could find some way to leave gaps in member space (perhaps
by making member index point to member groups with space for 4 or 8
member xids), and MultiXactIdExpand could create new multixacts that
point to the same member offset but a different size so they see extra
members, but that would also waste disk space, be hard to synchronize
and you'd need to fall back to copying the members into new member
space when the spare space is filled anyway.