On Mon, May 30, 2022 at 2:37 PM David Rowley <dgrowleyml@gmail.com> wrote:
> The results compare PG14 @ 0adff38d against master @ b3fb16e8b. In
> the chart, anything below 100% is a performance improvement over PG14
> and anything above 100% means PG15 is slower. You can see there's
> only the 64-byte / 64MB work_mem test that gets significantly slower
> and that there are only a small amount of other tests that are
> slightly slower. Most are faster and on average PG15 takes 90% of the
> time that PG14 took.
Shouldn't this be using the geometric mean rather than the arithmetic
mean? That's pretty standard practice when summarizing a set of
benchmark results that are expressed as ratios to some baseline.
If I tweak your spreadsheet to use the geometric mean, the patch looks
slightly better -- 89%.
--
Peter Geoghegan