Hi hackers!
I've prepared a patch with a 64-bit TOAST Value ID. It is a kind of prototype
and needs some further work, but it is already working and ready to play with.
I've introduced 64-bit value ID field in varatt_external, but to keep it compatible
with older bases 64-bit value is stored as a structure of two 32-bit fields and value
ID moved to be the last in the varatt_external structure. Also, I've introduced
different ID assignment function - instead of using global OID assignment function
I use individual counters for each TOAST table, automatically cached and after
server restart when new value is inserted into TOAST table maximum used value
is searched and used to assign the next one.
>Andres Freund:
>I think we'll need to do something about the width of varatt_external to make
>the conversion to 64bit toast oids viable - and if we do, I don't think
>there's a decent argument for staying with 4 byte toast OIDs. I think the
>varatt_external equivalent would end up being smaller in just about all cases.
>And as you said earlier, the increased overhead inside the toast table / index
>is not relevant compared to the size of toasted datums.
There is some small overhead due to indexing 64-bit values. Also, for smaller
tables we can use 32-bit ID instead of 64-bit, but then we would have a problem
when we reach the limit of 2^32.
Pg_upgrade seems to be not a very difficult case if we go keeping old TOAST
tables using 32-bit values,
Please have a look. I'd be grateful for some further directions.
GIT branch with this feature, rebased onto current master: