Now that a standby server can follow timeline switches through streaming
replication, we should do teach pg_receivexlog to do the same. Patch
attached.
I made one change to the way START_STREAMING command works, to better
support this. When a standby server reaches the timeline it's streaming
from the master, it stops streaming, fetches any missing timeline
history files, and parses the history file of the latest timeline to
figure out where to continue. However, I don't want to parse timeline
history files in pg_receivexlog. Better to keep it simple. So instead, I
modified the server-side code for START_STREAMING to return the next
timeline's ID at the end, and used that in pg_receivexlog. I also
modifed BASE_BACKUP to return not only the start XLogRecPtr, but also
the corresponding timeline ID. Otherwise we might try to start streaming
from wrong timeline if you issue a BASE_BACKUP at the same moment the
server switches to a new timeline.
When pg_receivexlog switches timeline, what to do with the partial file
on the old timeline? When the timeline changes in the middle of a WAL
segment, the segment old the old timeline is only half-filled. For
example, when timeline changes from 1 to 2, you'll have this in pg_xlog:
000000010000000000000006
000000010000000000000007
000000010000000000000008
000000020000000000000008
00000002.history
The segment 000000010000000000000008 is only half-filled, as the
timeline changed in the middle of that segment. The beginning portion of
that file is duplicated in 000000020000000000000008, with the
timeline-changing checkpoint record right after the duplicated portion.
When we stream that with pg_receivexlog, and hit the timeline switch,
we'll have this situation in the client:
000000010000000000000006
000000010000000000000007
000000010000000000000008.partial
What to do with the partial file? One option is to rename it to
000000010000000000000008. However, if you then kill pg_receivexlog
before it has finished streaming a full segment from the new timeline,
on restart it will try to begin streaming WAL segment
000000010000000000000009, because it sees that segment
000000010000000000000008 is already completed. That'd be wrong.
The best option seems to be to just leave the .partial file in place, so
as streaming progresses, you end up with:
000000010000000000000006
000000010000000000000007
000000010000000000000008.partial
000000020000000000000008
000000020000000000000009
00000002000000000000000A.partial
It feels a bit confusing to have that old partial file there, but that
seems like the most correct solution. That file is indeed partial. This
also ensures that if the server running on timeline 1 continues to
generate new WAL, and it fills 000000010000000000000008, we won't
confuse the partial segment with that name with a full one.
- Heikki