Sequential I/O Cost (was Re: A Better External Sort?)

On Wed, 2005-09-28 at 12:03 -0400, Ron Peacetree wrote:
> >From: "Jeffrey W. Baker" <jwbaker@acm.org>
> >Perhaps I believe this because you can now buy as much sequential I/O
> >as you want.  Random I/O is the only real savings.
> >
> 1= No, you can not "buy as much sequential IO as you want".  Even if
> with an infinite budget, there are physical and engineering limits.  Long
> before you reach those limits, you will pay exponentially increasing costs
> for linearly increasing performance gains.  So even if you _can_ buy a
> certain level of sequential IO, it may not be the most efficient way to
> spend money.

This is just false.  You can buy sequential I/O for linear money up to
and beyond your platform's main memory bandwidth.  Even 1GB/sec will
severely tax memory bandwidth of mainstream platforms, and you can
achieve this rate for a modest cost.

I have one array that can supply this rate and it has only 15 disks.  It
would fit on my desk.  I think your dire talk about the limits of
science and engineering may be a tad overblown.