Re: changeset generation v5-01 - Patches & git tree

On 2013-07-10 15:14:58 -0700, Kevin Grittner wrote:
> Andres Freund <andres@2ndquadrant.com> wrote:
> > Kevin Grittner <kgrittn@ymail.com> wrote:
> 
> >> (2)  An initial performance test didn't look very good.  I will be
> >> running a more controlled test to confirm but the logical
> >> replication of a benchmark with a lot of UPDATEs of compressed text
> >> values seemed to suffer with the logical replication turned on.
> >> Any suggestions or comments on that front, before I run the more
> >> controlled benchmarks?
> >
> > Hm. There theoretically shouldn't actually be anything added in that
> > path. Could you roughly sketch what that test is doing? Do you actually
> > stream those changes out or did you just turn on wal_level=logical?
> 
> It was an update of a every row in a table of 720000 rows, with
> each row updated by primary key using a separate UPDATE statement,
> modifying a large text column with a lot of repeating characters
> (so compressed well).  I got a timing on a master build and I got a
> timing with the patch in the environment used by
> test_logical_decoding.  It took several times as long in the latter
> run, but it was very much a preliminary test in preparation for
> getting real numbers.  (I'm sure you know how much work it is to
> set up for a good run of tests.)  I'm not sure that (for example)
> the synchronous_commit setting was the same, which could matter a
> lot.  I wouldn't put a lot of stock in it until I can re-create it
> under a much more controlled test.

So you didn't explicitly start anything to consume those changes?
I.e. using pg_receivellog or SELECT * FROM
start/init_logical_replication(...)?

Any chance there still was an old replication slot around?
SELECT * FROM pg_stat_logical_decoding;
should show them. But theoretically the make check in
test_logical_decoding should finish without one active...

> The one thing about the whole episode that gave me pause was that
> the compression and decompression routines were very high on the
> `perf top` output in the patched run and way down the list on the
> run based on master.

That's interesting. Unless there's something consuming the changestream
and the output plugin does something that actually requests
decompression of the Datums there shouldn't be *any* added/removed calls
to toast (de-)compression...
While consuming the changes there could be ReorderBufferToast* calls in
the profile. I haven't yet seem them in profiles, but that's not saying
all that much.

So:
> I don't have a ready explanation for that, unless your branch was
> missing a recent commit for speeding compression which was present on
> master.

It didn't have 031cc55bbea6b3a6b67c700498a78fb1d4399476 - but I can't
really imagine that making *such* a big difference. But maybe you hit
some sweet spot with the data?

Greetings,

Andres Freund

-- Andres Freund                       http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services