Re: Sync Rep: First Thoughts on Code

On Thu, 2008-12-11 at 19:19 +0900, Fujii Masao wrote:
> >
> > All I've asked for is the ability to turn on and turn back on archiving,
> > yes, with synchronisation so its safe.
> >(snip)

> OK, I will add such archiving feature. My new design of archiving is as follows.
> 
> Primary
> ----------
> I extend archive_mode as follows and make the user be able to choose the
> archiving strategy on the primary.
> 
> - always
>   The primary always archives the WAL. This is compatible with current (<=8.3)
>   archive_mode = on.
> 
> - none
>   The primary always doesn't archive the WAL. This is compatible with current
>   archive_mode = off.
> 
> - standalone
>   The primary doesn't archive the WAL only during replication. If replication is
>   not in progress, the primary archives the WAL. That is, the primary switches
>   the modes whenever replication starts / ends.
> 
>   [FLS->SLS]
>   When replication starts, the primary disable archiving *after* the switched
>   WAL file is archived. WAL streaming doesn't need to wait for disablement
>   of archiving, so the processing on the primary isn't blocked by starting of
>   replication. But, both WAL streaming and archiving would be in progress
>   for a while (until the switched WAL file is archived) after
> replication starts.

I'm OK with that, but that is slightly different from what Heikki had
said in relation to the point at which sync rep begins on primary, so he
may have a different view.

synchronous_replication means "if we a standby server has connected to
us we will wait for all WAL associated with a transaction to be
transferred prior to commit". So there is never a 100% guarantee that
the transaction is safe, just an "if possible, 100%".

So this implements the equivalent of DRBD Protocol A and B. Do we have
an option to allow the WALreceiver to fsync the WAL file after a commit
is received, which would make it equivalent to Protocol C? If we don't,
I'm OK with that since it reduces performance so much it isn't a
practical option in many cases. 
http://www.drbd.org/users-guide/s-replication-protocols.html

>   [SLS->FLS]
>   When replication starts, the primary restarts archiving immediately. This
>   also doesn't block the processing on the primary. But, this might cause
>   loss of some files from an archive if archiving is slow on the standby.
>   The primary should look for the last archived file (by the standby) from
>   an archive and restart archiving from the subsequent file? Of course,
>   the primary cannot archive it if it's already removed on the primary.

Standby will always have kept enough files to allow it to restart from
the last restartpoint, so a gap in the file sequence is unlikely. As
long as we archive the WAL file that contains the last LSN we
transferred before streaming failed. That conceivably might mean we need
to write a .ready message after a WAL file filled, which might mean we
have problems if the replication timeout is longer than the checkpoint
timeout, but that seems an unlikely configuration. And if anybody has a
problem with that we just recommend they use the "always" mode.

> Standby
> -----------
> I would add new option for achiving during recovery into recovery.conf
> (recovery_archive_mode). Though this option is similar to archive_mode,
> merging them would confuse the user more, I think. Or, I should merge?
> And, do you want to configure the archive command only for recovery?
> If so, I would add new option to specify the archive command during
> recovery (recovery_archive_command).

I think if you really want two archives or archiving during recovery
then this is desirable to avoid confusion.

Explaining all this in the docs will be fun. :-)

-- Simon Riggs           www.2ndQuadrant.comPostgreSQL Training, Services and Support