Re: pg_upgrade and rsync

* Andres Freund (andres@2ndquadrant.com) wrote:
> On 2015-01-23 14:05:10 -0500, Stephen Frost wrote:
> > * Andres Freund (andres@2ndquadrant.com) wrote:
> > > On 2015-01-23 13:52:54 -0500, Stephen Frost wrote:
> > > > That wouldn't actually help with what Bruce is trying to do, which
> > > > is to duplicate the results of the pg_upgrade from the master over to
> > > > the standby.
> > >
> > > Well, it'd pretty much obliviate the need to run pg_upgrade on the
> > > standby. As there's no renamed files you don't need to muck around with
> > > leaving hardlinks in place and such just so that rsync recognizes
> > > unchanged files.
> >
> > Uh, pg_upgrade always either creates a hard link tree or copies
> > everything over.
>
> Yes. The problem is that the filenames after pg_upgrade aren't the same
> as before. Which means that a simple rsync call won't be able to save
> anything because the standby's filenames differ.  What you can do is
> rsync both cluster directories (i.e. the old and the post pg_upgrade
> ones) and use rsync -H, right? Without transferring both -H won't detect
> the hardlinks as they need to be in the synced set. That's pretty
> cumbersome/complicated, and far from cheap.

The filenames don't need to be the same for rsync -H to work.  You
specifically do *not* want to independently rsync the old and new
clusters- you need to run a single rsync (and one for each tablespace)
with -H and then it'll realize that the old cluster on both systems is
identical and will just recreate the hard links, and copy the completely
new files (the catalog tables).

> > If I follow what you're suggesting, pg_upgrade would
> > need a new 'in-place' mode that removes all of the catalog tables from
> > the old cluster and puts the new catalog tables into place and leaves
> > everything else alone.
>
> No. Except that it'd preserve the relfilenodes (i.e. the filenames of
> relations) it'd work exactly the same as today. The standby is simply
> updated by rsyncing the new data directory of the primary to the
> standby.

You'd have to replace the existing data directory on the master to do
that, which pg_upgrade was designed specifically to not do, in case
things went poorly.  You'd still have to deal with the tablespace
directories being renamed also, since we include the major version and
catalog build in the directory name..

This whole process really isn't all that complicated in the end..

my_data_dir/old_cluster
my_data_dir/new_cluster

pg_upgrade
rsync -H --size-only my_data_dir/ standby:/path/to/my_data_dir
start the clusters
remove the old cluster on the master and standby.
Thanks,
    Stephen