Re: WIP: Fast GiST index build

On 12.07.2011 11:34, Alexander Korotkov wrote:
> New version of patch with a little more refactoring and comments.

Great! The README helps tremendously to understand this, thanks for that.

One thing that caught my eye is that when you empty a buffer, you load 
the entire subtree below that buffer, down to the next buffered or leaf 
level, into memory. Every page in that subtree is kept pinned. That is a 
problem; in the general case, the buffer manager can only hold a modest 
number of pages pinned at a time. Consider that the minimum value for 
shared_buffers is just 16. That's unrealistically low for any real 
system, but the default is only 32MB, which equals to just 4096 buffers. 
A subtree could easily be larger than that.

I don't think you're benefiting at all from the buffering that BufFile 
does for you, since you're reading/writing a full block at a time 
anyway. You might as well use the file API in fd.c directly, ie. 
OpenTemporaryFile/FileRead/FileWrite.

--   Heikki Linnakangas  EnterpriseDB   http://www.enterprisedb.com