> >> Are you filling multiple buffers in the buffer cache with a single
> >> read-call?
> >
> > yes, needs vector or ScatterGather IO.
>
> I would expect that to get only moderate improvement.
The vast improvement comes from 256k blocksize.
> To get
> the full benefit I would think you would want to either fire
> off a separate thread to do the read-ahead, use libaio, or
> funnel the read-ahead requests to a separate thread like our
> bgwriter only it would be a bgreader or something like that.
I like bgreader :-) But that looks even more difficult than grabbing 32
[scattered or contiguous] buffers at once.
Especially in a situation where there is no concurrent load it would be
nice to do CPU work while waiting for the next read ahead IO. If there
is enough parallel CPU load it is actually not so important. So I opt,
that on a high load server you get nearly all benefit without any sort
of aio.
> >> The OS should be doing readahead for us anyway, so I don't see how
> >> just issuing multiple ReadBuffers one after each other helps.
> >
> > Last time I looked OS readahead was only comparable to 32k blocked
reads.
> > 256k blocked reads still perform way better. Also when the OS is
> > confronted with an IO storm the 256k reads perform way better than
OS readahead.
>
> Well that's going to depend on the OS. Last I checked Linux's
> readahead logic is pretty straightforward and doesn't try to
> do any better than 32k readahead and is easily fooled.
> However I wouldn't be surprised if that's changed.
My test was on AIX, 32 or 64k seem quite common, at least as default
setting.
Also on some OS's (like HPUX) OS readahead and writebehind strategy
changes with large IO blocksizes, imho beneficially.
Andreas