Re: ZFS prefetch considered evil?

On Wed, 8 Jul 2009, Yaroslav Tykhiy wrote:

> My conclusion is that although ZFS prefetch is supposed to be adaptive and
> handle random access more or less OK, in reality there is plenty of room for
> improvement, so to speak, and for now Postgresql performance can benefit from
> its staying just disabled.  The same may apply to other database systems as
> well.

Yup; this is even spelled out at
http://www.cuddletech.com/blog/pivot/entry.php?id=1040

"...the most common complain tends to be by databases which strictly work
in fixed 8K blocks and manage their own caches very effectively. If you
think you have such a case, file-level prefetch can be tuned on the fly
using mdb, I encourage you to play with it and see what is best for your
workload..."

Anecdotal reports (which never seem to have repeatable test cases sadly)
abound about prefetch issues:

http://southbrain.com/south/2008/04/the-nightmare-comes-slowly-zfs.html
http://unix.derkeiler.com/Mailing-Lists/FreeBSD/current/2007-06/msg00671.html

Also, there was a pretty serious ZFS problem in this area that got fixed
in the middle of last year on Solaris.  Your FreeBSD install might be
based on a build that is using the older, known bad logic here.  See
http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide#Device-Level_Prefetching
and http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6437054 for
details.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD