Re: lock contention on parallel COPY ?

On Fri, 2008-09-26 at 12:38 -0400, Tom Lane wrote:
> Stefan Kaltenbrunner <stefan@kaltenbrunner.cc> writes:
> > heh no log archiving - I actually said that I'm now playing with 
> > --truncate-before-load which seems to cause a noticeable performance (as 
> > in IO generated) increase but I still see >130000 context switches/s and 
> > a profile that looks like:
> 
> > samples  %        symbol name
> > 55526    16.5614  LWLockAcquire
> > 29721     8.8647  DoCopy
> > 26581     7.9281  CopyReadLine
> > 25105     7.4879  LWLockRelease
> > 15743     4.6956  PinBuffer
> > 14725     4.3919  heap_formtuple
> 
> Still a lot of contention for something, then.  You might try turning on
> LWLOCK_STATS (this only requires recompiling storage/lmgr/lwlock.c) to
> get some evidence about what.

Probably loading a table with a generated PK or loading data in
ascending sequence, so its contending heavily for the rightmost edge of
the index.

We need to load data a block at a time and buffer the inserts into the
index also, so we don't need to lock/unlock per row.

-- Simon Riggs           www.2ndQuadrant.comPostgreSQL Training, Services and Support