> If I thought that pgbench was representative of anything, or even
> capable of reliably producing repeatable numbers, then I might subscribe
> to results derived this way. But I have little or no confidence in
> pgbench. Certainly I don't see how you'd use it to produce
> recommendations for a range of application scenarios, when it's only
> one very narrow scenario itself.
Sigh. People always complain "pgbench does not reliably producing
repeatable numbers" or something then say "that's because pgbench's
transaction has too much contention on the branches table". So I added
-N option to pgbench which makes pgbench not to do any UPDATE to
the branches table. But still people continue to complian...
There should be many factors that would produce non-repeatable
results exist, for instance kenel buffer, PostgreSQL's buffer manager,
pgbench itself etc. etc... So far it seems no one has ever made clean
explanation why non-repeatable results happen...
--
Tatsuo Ishii