Re: Replication identifiers, take 3

On 2 October 2014 09:49, Heikki Linnakangas  wrote:

>> What I've previously suggested (and which works well in BDR) is to add
>> the internal id to the XLogRecord struct. There's 2 free bytes of
>> padding that can be used for that purpose.
>
>
> Adding a field to XLogRecord for this feels wrong. This is for *logical*
> replication - why do you need to mess with something as physical as the WAL
> record format?

Some reasons why this would be OK:

1. We've long agreed that adding information to the WAL record is OK,
just as long as it doesn't interfere (much) with the primary purpose.
It seems OK to change wal_level so it is a list of additional info,
e.g. wal_level = 'hot standby, logical, originid', or just wal_level =
'$new_level_name' that includes originid info

2. We seem to have agreed elsewhere that extensions to WAL will be
allowed. So perhaps redefining those bytes is something that can be
re-used. That way we don't all have to agree what we'll use them for.
Just call a user defined function that returns a 2 byte integer, or
zero if no plugin.

3. As for how many bytes there are - I count 6 wasted bytes on a
MAXALIGN=8 record. Plus we discussed long ago ways we can save another
4 bytes on records that don't have a backup block, since in that case
xl_tot_len == xl_len. I've also got a feeling that WAL records that
are 2 billion bytes long might be overkill. I figure we could easily
make a length limit of something less than that - only commit records
can be longer than 2^19 bytes when they have >65536 subxids, which is
hardly common. Plus xl_prev is always at least 4 byte aligned, so
there are at least 3 bits to be reclaimed from there. (Plus we have 7
unused RmgrId bits, though maybe we want to keep those)

So I count about 14 bytes of potentially reclaimable space in the WAL
record header, or 10 with backup blocks. The reason we never reclaimed
it before was that benchmarks previously showed that reducing the
volume of WAL had little effect on performance, we weren't looking
specifically at WAL volume or info content. (And the perf result is
probably different now anyway).

Should we grovel around to reclaim any of that? Not necessary, the
next person with a good reason to use some space can do that.

Pick any of those: I see no reason to prevent reusing 2 bytes for a
useful purpose, when we do all agree it is a useful purpose.

-- Simon Riggs                   http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services