From: Claudio Freire Sent: Friday, September 02, 2016 1:27 PM
On Thu, Sep 1, 2016 at 11:30 PM, Mike Sofen <msofen@runbox.com> wrote:
> It's obvious the size of the batch exceeded the AWS server memory,
> resulting in a profoundly slower processing time. This was a true,
> apples to apples comparison between Pass 1 and Pass 2: average row
> lengths were within 7% of each other (1121 vs 1203) using identical
> table structures and processing code, the only difference was the target server.
>
> I'm happy to answer questions about these results.
Are you sure it's a memory thing and not an EBS bandwidth thing?
EBS has significantly less bandwidth than direct-attached flash.
You raise a good point. However, other disk activities involving large data (like backup/restore and pure large table copying), on both platforms, do not seem to support that notion. I did have both our IT department and Cisco turn on instrumentation for my last test, capturing all aspects of both tests on both platforms, and I’m hoping to see the results early next week and will reply again.
Mike