I have just realized that the recent patches in pgbench have altered its
behavior in a way that destroys reproducibility of results --- I'm
seeing reported TPS numbers about twice what they were before that.
I'd love to say we did something in the past month that made the backend
2X faster, but sadly, tain't so :-(
I've only done limited investigation but what seems to be happening
is that the "-s" scale factor is no longer being applied to adjust the
range of the random variables, leading to only part of the database
being touched. For example, with -s = 10, the older code updates all 10
"branches" rows but CVS HEAD only seems to touch bid = 1.
I see that the code still thinks it is scaling the numbers, but the
mechanism for doing that has changed drastically, and I suspect that's
the problem. I assume there's some garden-variety bug in there.
At a more fundamental level, the recent patches seem to mean that
pg_dump is doing noticeably more work to issue its queries than it was
before, and I wonder whether this isn't going to destroy cross-version
reproducibility in its own way. Have you done any timing studies to
verify that the new and old code have comparable performance? I don't
mind enhancing pgbench's functionality for non-default tests, but I
think it's important that the default case remain comparable over time.
If there is a noticeable speed difference then I'd vote for going back
to the klugy old code for adjusting the default script's values.
And at the nitpicking level, is ":tps" a good name for the variable that
reflects the scaling factor? It seems awfully easy to confuse that with
the TPS numbers that pgbench reports. Perhaps ":scale" or some such
would be better.
regards, tom lane