I think Nikhil was going to invoke the rsync black magic as described inhttps://www.postgresql.org/docs/current/pgupgrade.html, not for copying the entire PGDATA from the master to standbys. While the rsync magic should be hundreds of times faster because only modified files will be copied and the majority of files will be just hardlinked, I've never felt I have enough skill and luck to undertake this method.
Yes, I am using rsync only to copy modified files.
Because pg_upgrade basically runs initdb for you. You are dealing with a new cluster after pg_upgrade, in fact.
Thanks for the details Victor and Julien.
Thanks,
Nikhil
On Tue, Feb 8, 2022 at 2:15 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Tue, Feb 08, 2022 at 08:32:22AM +0000, Victor Sudakov wrote: > Julien Rouhaud wrote: > > > > > pg_basebackup takes time for large databases (> 5TB). I feel rsync should > > > be faster. > > > > Not necessarily. pg_basebackup will do simple sequential read of all the data, > > which is probably the fastest thing to do. rsync will do some extra processing > > that isn't required in that scenario, so it's likely to be slower (although > > probably only marginally slower). > > I think Nikhil was going to invoke the rsync black magic as described > in https://www.postgresql.org/docs/current/pgupgrade.html, not for > copying the entire PGDATA from the master to standbys. While the rsync > magic should be hundreds of times faster because only modified files > will be copied and the majority of files will be just hardlinked, I've > never felt I have enough skill and luck to undertake this method.
Ah, if that's the case I don't think there's any doubt to have for rsync being faster. And this method works just fine if you take care of what you're doing.