2011/11/16 Scott Ribe <scott_ribe@elevated-dev.com>:
> On Nov 16, 2011, at 10:11 AM, Jean-Armel Luce wrote:
>
>> You are right. I used -a, and I was wanting to be more meaningful so I wrote --all in my post.
>> Please read --archive insterad of --all
>
> Oh, OK. Still seems odd that it took so much longer. Granted, for the files with different timestamps but identical
contents,it then syncs them. But it does so by checksumming blocks, comparing checksums, and sending only blocks that
aredifferent over the network. Granted, it has to send some checksums over the network, but that's pretty minor
traffic.I believe you said you'd seen 125Mb/s over the network? Is that actually accurate? Does the network connection
havehigh latency?
>
> Also, I believe you said -z seemed to slow it down? That has not been my experience at all with rsync'ing pg
databases.Between all the values that are stored as plain text, and the redundancies in indexes, I usually see a good
speedincrease from compressing the data in transit.
>
> I'm certainly glad that you've got a 3x speed increase--that's significant progress. But still, something seems odd
aboutthe performance you've reported, so I'm left wondering about what could cause that. Disk performance glitch at one
endor the other, network performance, CPU load???
checksum are calculated for 1024 bytes(maybe 2048) per default, for
100GB that makes a huge number of checksum calculated for nothing. It
is possible to increase the block-size (to 8kb for example) for the
checksum but it also increase the risk of false positive (I am not
aware if it is possible to provide the checksum algorithm and size
expected to rsync, or if there can be md5sum collision on 8Kb data).
>
> One thing worth doing I think is to use --stats on every test, so you can see every time how many files and how much
datais actually transferred. Also, if you're sitting there watching, sometimes --progress can be informative...
>
> --
> Scott Ribe
> scott_ribe@elevated-dev.com
> http://www.elevated-dev.com/
> (303) 722-0567 voice
>
>
>
>
>
--
Cédric Villemain +33 (0)6 20 30 22 52
http://2ndQuadrant.fr/
PostgreSQL: Support 24x7 - Développement, Expertise et Formation