On Wednesday, June 20, 2012 03:24:58 PM Robert Haas wrote:
> On Wed, Jun 20, 2012 at 9:12 AM, Andres Freund <andres@2ndquadrant.com>
wrote:
> > Uh. I don't want to just go around and replace anything randomly.
> > Actually I don't want to change anything for now except whats necessary
> > to get the patch in. The point I tried to make was just that the
> > relatively widespread usage of similar structure make it likely that it
> > can be used in more places in future.
>
> Well, the question is for anywhere you might be thinking of using
> this: why not just use List? We do that in a lot of other places, and
> there's not much reason to event something new unless there is a
> problem with what we already have. I assume this is related to
> logical replication somehow, but it's not clear to me exactly what
> problem you hit doing this in the obvious way.
It incurs a rather high performance overhead due to added memory allocations
and added pointer indirections. Thats fine for most of the current users of
the List interface, but certainly not for all. In other places you cannot even
have memory allocations because the list lives in shared memory.
E.g. in the ApplyCache, where I use the submitted ilist.h stuff, when
reconstructing transactions you add to a potentially really long linked list
of individual changes for every interesting wal record. Before I prevented
memory allocations in that path it took about 12-14% of the time when applying
changes in the same backend. Afterwards it wasn't visible in the profile
anymore.
Several of the pieces of code I pointed out in a previous email use open-coded
list implementation exactly to prevent those problems.
If you look at the parsing, planning & execution of trivial statements you
will also notice the overhead of memory allocations. A good bit of those is
caused by list manipulation. Check Stephen Frost's "Pre-alloc ListCell's
optimization" for workarounds..
Andres
-- Andres Freund http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services