Re: Doubt in pgbench TPS number

Hello Tom,

> FWIW, I vote with Tatsuo-san.  Such a change will break comparability of
> results

I would not classify a performance measure as a "result compatibility" 
issue. What matters is the *correction* of the results.

When a bug is fixed anywhere in pg it may change performance significantly 
for some tests, and I have never seen this as a relevant consideration not 
to fix a problem...

> with all previous versions, which means it's not something to do
> in minor releases, even if we now believe the previous results were
> somewhat bogus.  Arguing that it's "not a behavioral change" seems
> quite loony to me: for most people, the TPS numbers are the only
> interesting output of pgbench.

I think that if people are interested in tps without reconnecting on each 
transaction they would not run with "-C" to trigger reconnections and then 
look at the "tps without connections" for the figure they want... so I do 
not think that keeping this error is worth anything.

On the other hand, and on the principle, keeping the bug looks wrong. I 
cannot agree with the logic behind shipping something which is known bad, 
especially to display *optimistic* performances... It looks too much like 
the VW way:-)

> I think we should fix it in 9.5 and document it as an incompatible 
> change.

Hmm.

> (Note: I've not read the patch, so this is not an opinion about its
> correctness.)

That is another question:-)

-- 
Fabien.