How does that readahead tunable affect random reads or mixed random / sequential situations? In many databases, the worst case scenarios aren't when you have a bunch of concurrent sequential scans but when there is enough random read/write concurrently to slow the whole thing down to a crawl. How the file system behaves under this sort of concurrency
I would be very interested in a mixed fio profile with a "background writer" doing moderate, paced random and sequential writes combined with concurrent sequential reads and random reads.
I've started to display the effects of changing the Linux block device readahead buffer to the sequential read performance using fio.
Ah ha, told you that was your missing tunable. I'd really like to see the whole table of one disk numbers re-run when you get a chance. The reversed ratio there on ext2 (59MB read/92MB write) was what tipped me off that something wasn't quite right initially, and until that's fixed it's hard to analyze the rest.
Based on your initial data, I'd say that the two useful read-ahead settings for this system are 1024KB (conservative but a big improvement) and 8192KB (point of diminishing returns). The one-disk table you've got (labeled with what the default read-ahead is) and new tables at those two values would really flesh out what each disk is capable of.