On 7/3/2014 10:13 PM, Bosco Rama wrote:
> Is the issue with S3 or just transfer time? I would expect that
> 'rsync' with the '--partial' option (or -P if you want progress info
> too) may help there.
Don't know if rsync and S3 work together or what that would mean, but
it's not an issue I'm suffering now. I do think they may now have a
multipart upload with s3cmd (which I use), though that also wasn't
available when we first built our scripts.
I suspect nothing is really helping here and I'm mostly limited by disk
I/O, but not sure why the pg_dump is so much slower than pg_restore as
they are all on the same disks. I say this because even with pg_dump
-Z0 | gpg -z 0 and gzip removed entirely and no --oids on pg_dump,
there's no effective difference in overall speed. While I can see all
of those processes vying for resources via 'top -c', the throughput
remains much the same.