On Tue, 15 Apr 2008, Florian Weimer wrote:
> * Jeff:
>
>> Using 4 of these with a dataset of about 30GB across a few files
>> (Machine has 8GB mem) I went from around 100 io/sec to 330 changing to
>> noop. Quite an improvement. If you have a decent controller CFQ is
>> not what you want. I tried deadline as well and it was a touch
>> slower. The controller is a 3ware 9550sx with 4 disks in a raid10.
>>
>> I'll be trying this out on the big array later today. I found it
>> suprising this info wasn't more widespread (the use of CFQ on a good
>> controller).
>
> 3ware might be a bit special because the controller has got very deep
> queues on its own, so many assumptions of the kernel I/O schedulers do
> not seem to apply. Toying with the kernel/controller queue depths
> might help, but I haven't done real benchmarks to see if it's actually
> a difference.
>
> A few days ago, I experienced this: On a machine with a 3ware
> controller, a simple getxattr call on a file in an uncontended
> directory took several minutes because a PostgreSQL dump process was
> running in the background (and some other activity of a legacy
> database which caused frequent fdatasync calls). Clearly, this is
> unacceptable, and I've since switched to the deadline scheduler, too.
> So far, this particular behavior hasn't occurred again.
one other thing to watch out for. up until very recent kernels (2.6.23 or
2.6.24) it was possible for one very busy block device to starve other
block devices. they added isolation of queues for different block devices,
but I've seen reports that the isolation can end up throttling high
performance devices as a result. I haven't had time to really dig into
this, but there are tuning knobs available to adjust the que space
available to different devices and the reports are significantly better
activity on a tuned system.
David Lang