Re: [HACKERS] Clarification in pg10'spgupgrade.html step 10 (upgrading standby servers)

On Sat, Sep 16, 2017 at 11:36:40PM +0200, Andreas Joseph Krogh wrote:
> På lørdag 16. september 2017 kl. 18:34:51, skrev Bruce Momjian <
> bruce@momjian.us>:
>     No.  If you ran initdb with --waldir on the new primary, you will create
>     a symbolic link in the PGDATA directory, and a directory outside of
>     PGDATA for storing the WAL.  When you run rsync on the new primary
>     PGDATA directory, you will copy the symlink in the PGDATA directory, but
>     it will point to probably nothing on the standby.
>
>  
>  
> The misunderstanding here comes from the fact that I used pg_upgradecluster
> like this:
> pg_upgradecluster --method=upgrade --link 9.6 main
>  
> and it didn't issue initdb with --waldir on the new cluster (because
> pg_upgradecluster issues initdb for you), so pg_wal ended up in $PGDIR because
> pg_upgradecluster didn't figure out the old cluster was initialized with
> --xlogdir. This is why I thought i made sense mentioning that one had to move
> pg_wal manually.
>  
> I know it's debian-stuff, but maybe it's worth mentioning pg_upgradecluster
> somewhere and recommend not using it? It seems to start the new cluster
> automatically and when upgrading standbys section 10 tells you not to do that.

So you didn't really follow the instructions, but instead are trying to
use the standby part of the instructions and found a problem with the
way pg_upgradecluster handled it.  We really can't document this.

It would be good to report the bug to pg_upgradecluster developers
though.

Yes, I can see rsync not working that case.

>     > should be clearly pointed out that copying pg_wal is only needed in those
>     > cases, and that it can be done with whatever network-copying procedure
>     you're
>     > familiar with, ie. scp/rsync. This step is not similar to the steps
>     required
>     > for copying tablespaces outside $PGDATA, so it's worth documenting
>     explicitly.
>     > Maybe also telling users to ensure the synlink (in $PGDATA) to pg_wal on
>     > standby points to pg_wal.
>
>     Why tell them new instructions when the rsync instructions work fine?
>     What is the value?
>
>  
> The rsync instructions mentioned in 10.F all address the --link scenario and
> use "--delete --hard-links --size-only", and "merge 2 source-dirs into one",
> which isn't relevant when copying pg_wal.
>  
> This sentence:
> "If you have relocated pg_wal outside the data directories, rsync must be run
> on those directories too."
> implies one must follow the rsync pattern elsewhere in 10.F, which isn't really
> true. Maybe re-wording it to:
> "If you have relocated pg_wal outside the data directories you must copy it
> over to the new standby, and ensure the symlink from $PGDATA points to it"
> helps?

We can't document every possible configuration, especially if a
secondary tool is used in the middle.