Re: 2018-03 Commitfest Summary (Andres #1)

Hello Peter,

>> On the "adequate return" point, my opinion is that currently pgbench is just
>> below the feature set needed to be generally usable, so not improving it is
>> a self-fullfilling ensurance that it will not be used further. Once the
>> "right" feature set is reached (for me, at least extracting query output
>> into variables, having conditionals, possibly a few more functions if some
>> benches use them), whether it would be actually more widely used by both
>> developers and users is an open question.
>
> FWIW, I think that pgbench would become a lot more usable if someone
> maintained a toolset for managing pgbench. Something similar to Greg
> Smith's pgbench-tools project, but with additional features for
> instrumenting the server. There would be a lot of value in integrating
> it with third party tooling, such as perf and BCC, and in making it
> easy for non-experts to run relevant, representative tests.
>
> Things like the rate limiting and alternative distributions were
> sorely needed, but there are diminishing returns. It's pretty clear to
> me that much of the remaining low hanging fruit is outside of pgbench
> itself.

It happens that I might start something on the line of what you are 
suggesting above.

However there is a minimal set of features needed in pgbench itself, 
especially on the scripting side (functions, variables, conditions, error 
handling... which are currently work in progress). I do not think that it 
would be make any sense to re-implement all detailed data collection, load 
throttling, client & thread handling outside pgbench, just because there 
is a missing basic feature such as a particular hash function or a stupid 
\if on the result of a query to implement a simple benchmark.

> None of the more recent pgbench enhancements seem to make it easier to 
> use.

I agree that "easier to use" is a worthy objective, and that its answer is 
probably partly outside pgbench (although there could be parts inside, eg 
json/CSV/... outputs to help collect performance data for instance).

-- 
Fabien.