Re: [PATCH] pgbench --throttle (submission 7 - with lag measurement)

Hello Tatsuo,

> Now I have question regarding the function.
>
> ./pgbench -p 5433 -S -T 10 -R 10000 test
> tps = 7133.745911 (including connections establishing)
>
> What does "average rate limit lag" mean? From the manual:
> [...]
> So in my understanding the number shows the delay time before *each*
> transaction starts.

... with respect to the schedule time assigned by the rate-limiting 
stochastic process. This is to detect that rate limiting does not work 
properly.

> If my understanding is correct, why 71339 (total transactions) * 862.534 
> ms = 61532 sec could exceed 10 seconds, which is the total run time?

It is possible, because each transaction is far behind schedule, and you 
cumulate the lateness.

Say you have transactions schedules every 0.1 second, but they take 2 
second to complete:
 1. scheduled at 0.0, start at 0.0 2. scheduled at 0.1, start at 2.0, 1.9 second lag 3. scheduled at 0.2, start at 4.0,
3.8second lag, cumulative lag 5.7 s 4. scheduled at 0.3, start at 6.0, 5.7 second lag, cumulative lag 11.4 s 5.
scheduledat 0.4, start at 8.0, 7.6 second lag, cumulative lag 19.0 s 6. scheduled at 0.5, never starts
 

If we stop at 10.0 seconds, 5 transaction have been processed, the average 
lag is about 3.8 seconds, the cumulative lag is 19.0 seconds. The lag of a 
given transaction can cover lag from previous ones.

Basically, if the lag is anything but small, it means that the database 
cannot handle the load and that something is amiss. In your example you 
required 10000 tps, but the database can only handle 7000 tps.

Note that the database could catchup at some point, say it usually can 
handle more that 10000 tps, but while the database dump is running it 
falls far behing schedule, and then one the dump is done it goes back to 
nominal and "late" transactions are finally processed. The max lag would 
show that something was amiss during the bench, even if the average lag
is quite low.

> Also I noticed small bug.
>
> ./pgbench -R 0 test
> invalid rate limit: 0
>
> Shouldn't this be treated as if -R is not specified? Actually in the program:
>
> /*
> * When threads are throttled to a given rate limit, this is the target delay
> * to reach that rate in usec.  0 is the default and means no throttling.
> */
> int64        throttle_delay = 0;
>
> So it seems treating "-R 0" means "no throttling" makes more sense to me.

Note that the rate is expressed in tps which make sense to users, but the 
internal variable is in usec which is more useful for scheduling, and is 
the inverse of the other.

So -R 0 would mean zero tps, that is an infinite delay, but a 0 delay 
would require an infinite tps. As requiring 0 tps does not make sense, I 
decided to disable that. If you really fill that "-R 0" should mean 
disable the feature, I'm fine with that, but this is not exactly logical 
wrt tps.

-- 
Fabien.