On Wed, 25 Jul 2007, Tom Lane wrote:
> Gentoo always leaves me wondering exactly what I'm running today,
> and I think reproducibility is an important attribute for a benchmarking
> machine.
At this point, there's enough performance variations even between
individual Linux kernel releases that I'm not sure how much
reproducibility you're ever going to get here. Are the differences
between Gentoo and RHEL any bigger than those, say, between RHEL and SuSE?
The idea of setting this up with a long-term stable distribution runs
counter to one of the things that I think is important to explore here,
which is testing how more recent Linux kernels have improved their
scalability. Do you really want to put a lot of time into identifying and
working around the source of a problem with the typically older kernels
that ship with the more stable releases if one answer is "that goes away
if you use 2.6.21 or later because they fixed the bug that caused it"?
I've watched that sort of thing happen with PG+Linux, and when involved in
one of the recent roving talks Gavin mentioned I recall him mentioning a
bit of that experience himself. You'd be hard pressed to find a better
platform for that kind of experimentation than Gentoo.
The best you can hope for, I think, is that you can walk away with some
general benchmark expectations and "on Gavin's machine, this worked
better"; then try to replicate that improvement elsewhere. If you want to
push bleeding edge performance, I'd expect it's impractical to do that and
target long-term results stability at the same time.
--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD