Re: optimizing CleanupTempFiles

On Wed, 2008-09-17 at 16:25 -0400, Alvaro Herrera wrote:

> We've been profiling a large system (8 CPUs, 64 GB of memory, some
> dozens of disks) which seems rather more swamped than it should.  Part
> of the problem seems to come from CleanupTempFiles, the second entry in
> oprofile output.

I'm glad you've observed this also. I saw it about two years ago but
wasn't able to convince anyone else it existed at the time.

> I can see two simple ways to solve this problem.  One is to create a
> second array that keeps pointers to the temp files in VfdCache.  Then,
> on CleanupTempFiles we scan that array instead of VfdCache directly.
> 
> The second one is to use a separate VfdCache for temp files, but this
> seems much more involved, requiring touch almost all of fd.c.
> 
> Of course, perhaps there's another solution which involves rethinking
> fd.c in a more thorough fashion, but I'm not sure how right offhand.

Simple solution is to have a state variable so you can see whether a
backend has created an temp files in this transaction. Most don't, so I
think the above two solutions are overkill. If we created any, scan for
them, if not, don't. Just a simple boolean state, just as we have for
AtEOXact_RelationCache().

-- Simon Riggs           www.2ndQuadrant.comPostgreSQL Training, Services and Support