Re: add modulo (%) operator to pgbench

Hello Heikki,

>> If you reject it, you can also remove the gaussian and exponential random
>> distribution which is near useless without a mean to add a minimal
>> pseudo-random stage, for which "(x * something) % size" is a reasonable
>> approximation, hence the modulo submission.
>
> I'm confused. The gaussian and exponential patch was already committed.

Yes.

They are significant patches that really involved significant work, and 
which are only really useful with a complementary stupid 10 lines patch 
which is being rejected without understanding why it is needed.

> Are you saying that it's not actually useful, and should be reverted? 
> That doesn't make sense to me, gaussian and exponential distributed 
> values seems quite useful to me in test scripts.

Yes and no.

Currently these distributions are achieved by mapping a continuous 
function onto integers, so that neighboring integers get neighboring 
number of draws, say with size=7:
  #draws     10 6 3 1 0 0 0  // some exponential distribution  int drawn   0 1 2 3 4 5 6

Although having an exponential distribution of accesses on tuples is quite 
reasonable, the likelyhood there would be so much correlation between 
neighboring values is not realistic at all. You need some additional 
shuffling to get there.

> I don't understand what that pseudo-random stage you're talking about is. Can 
> you elaborate?

The pseudo random stage is just a way to scatter the values. A basic 
approach to achieve this is "i' = (i * large-prime) % size", if you have a 
modulo. For instance with prime=5 you may get something like:
  #draws     10 6 3 1 0 0 0  int drawn   0 1 2 3 4 5 6 (i)  scattered   0 5 3 1 6 4 2 (i' = 5 i % 7)

So the distribution becomes:
  #draws     10 1 0 3 0 6 0  scattered   0 1 2 3 4 5 6

Which is more interesting from a testing perspective because it removes 
the neighboring value correlation.

A better approach is to use a hash function. "i' = hash(i) % size",
although it skews the distribution some more, but the quality of the 
shuffling is much better than with the mult-modulo version above.
Note that you need a modulo as well...

I must say that I'm appaled by a decision process which leads to such 
results, with significant patches passed, and the tiny complement to make 
it really useful (I mean not on the paper or on the feature list, but in 
real life) is rejected...

-- 
Fabien.