Re: 8.4 open item: copy performance regression?

On Sun, 2009-06-21 at 20:37 +0300, Heikki Linnakangas wrote:
> Robert Haas wrote:
> > On Sun, Jun 21, 2009 at 11:52 AM, Tom Lane<tgl@sss.pgh.pa.us> wrote:
> >> So to my mind, the only question left to answer (at least for the 8.4
> >> cycle) is "is 16MB enough, or do we want to make the ring even bigger?".
> >> Right at the moment I'd be satisfied with 16, but I wonder whether there
> >> are scenarios where 32MB would show a significant advantage.
> > 
> > Even 32MB is not that much.  It seems to me that in any realistic
> > production scenario you're going to have at least half a gig of shared
> > buffers, so we're really talking about at most one-sixteenth of the
> > shared buffer arena, and possibly quite a bit less.  I think that's
> > pretty conservative.
> 
> I was going to say that since we flush the WAL every 16MB anyway (at 
> every XLOG file switch), you shouldn't see any benefit with larger ring 
> buffers, since to fill 16MB of data you're not going to write more than 
> 16MB WAL. But then I realized that that's not true if you have an 
> unusually low fillfactor. If you only fill each page say 50% full, 
> you're going to use 32MB worth of data pages but only write 16MB of WAL. 
> And maybe you could have a situation like that with very wide rows as 
> well, with wasted space on each page that's not enough to store one more 
>   row.

If walwriter is working correctly then it should be writing and fsyncing
WAL, while the COPY process just inserts WAL. I don't see that as an
argument to limit us to 16MB. But I take your point as being an argument
in favour of that as a consensus value for us to choose.

-- Simon Riggs           www.2ndQuadrant.comPostgreSQL Training, Services and Support