Re: [PATCH] pgbench --throttle (submission 7 - with lag measurement)

> Hello Tatsuo,
> 
>> Now I have question regarding the function.
>>
>> ./pgbench -p 5433 -S -T 10 -R 10000 test
>> tps = 7133.745911 (including connections establishing)
>>
>> What does "average rate limit lag" mean? From the manual:
>> [...]
>> So in my understanding the number shows the delay time before *each*
>> transaction starts.
> 
> ... with respect to the schedule time assigned by the rate-limiting
> stochastic process. This is to detect that rate limiting does not work
> properly.
> 
>> If my understanding is correct, why 71339 (total transactions) *
>> 862.534 ms = 61532 sec could exceed 10 seconds, which is the total run
>> time?
> 
> It is possible, because each transaction is far behind schedule, and
> you cumulate the lateness.
> 
> Say you have transactions schedules every 0.1 second, but they take 2
> second to complete:
> 
>  1. scheduled at 0.0, start at 0.0
>  2. scheduled at 0.1, start at 2.0, 1.9 second lag
>  3. scheduled at 0.2, start at 4.0, 3.8 second lag, cumulative lag 5.7
>  s
>  4. scheduled at 0.3, start at 6.0, 5.7 second lag, cumulative lag 11.4
>  s
>  5. scheduled at 0.4, start at 8.0, 7.6 second lag, cumulative lag 19.0
>  s
>  6. scheduled at 0.5, never starts
> 
> If we stop at 10.0 seconds, 5 transaction have been processed, the
> average lag is about 3.8 seconds, the cumulative lag is 19.0
> seconds. The lag of a given transaction can cover lag from previous
> ones.
> 
> Basically, if the lag is anything but small, it means that the
> database cannot handle the load and that something is amiss. In your
> example you required 10000 tps, but the database can only handle 7000
> tps.
> 
> Note that the database could catchup at some point, say it usually can
> handle more that 10000 tps, but while the database dump is running it
> falls far behing schedule, and then one the dump is done it goes back
> to nominal and "late" transactions are finally processed. The max lag
> would show that something was amiss during the bench, even if the
> average lag
> is quite low.

Thanks for detailed explainations. I now understand the function.

>> Also I noticed small bug.
>>
>> ./pgbench -R 0 test
>> invalid rate limit: 0
>>
>> Shouldn't this be treated as if -R is not specified? Actually in the
>> program:
>>
>> /*
>> * When threads are throttled to a given rate limit, this is the target
>> * delay
>> * to reach that rate in usec.  0 is the default and means no throttling.
>> */
>> int64        throttle_delay = 0;
>>
>> So it seems treating "-R 0" means "no throttling" makes more sense to
>> me.
> 
> Note that the rate is expressed in tps which make sense to users, but
> the internal variable is in usec which is more useful for scheduling,
> and is the inverse of the other.
> 
> So -R 0 would mean zero tps, that is an infinite delay, but a 0 delay
> would require an infinite tps. As requiring 0 tps does not make sense,
> I decided to disable that. If you really fill that "-R 0" should mean
> disable the feature, I'm fine with that, but this is not exactly
> logical wrt tps.

Ok, your statement seems to be fair. Unless someone complains the
point, I leave it as it is.

I'm going to test your patches on Mac OS X and Windows.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp