Re: 2nd Level Buffer Cache

Merlin Moncure  Monday 21 March 2011 20:58:16
> On Mon, Mar 21, 2011 at 2:08 PM, Greg Stark  wrote:
> > On Mon, Mar 21, 2011 at 3:54 PM, Merlin Moncure  
wrote:
> >> Can't you make just one large mapping and lock it in 8k regions? I
> >> thought the problem with mmap was not being able to detect other
> >> processes
> >> (http://www.mail-archive.com/pgsql-general@postgresql.org/msg122301.htm
> >> l) compatibility issues (possibly obsolete), etc.
> > 
> > I was assuming that locking part of a mapping would force the kernel
> > to split the mapping. It has to record the locked state somewhere so
> > it needs a data structure that represents the size of the locked
> > section and that would, I assume, be the mapping.
> > 
> > It's possible the kernel would not in fact fall over too badly doing
> > this. At some point I'll go ahead and do experiments on it. It's a bit
> > fraught though as it the performance may depend on the memory
> > management features of the chipset.
> > 
> > That said, that's only part of the battle. On 32bit you can't map the
> > whole database as your database could easily be larger than your
> > address space. I have some ideas on how to tackle that but the
> > simplest test would be to just mmap 8kB chunks everywhere.
> 
> Even on 64 bit systems you only have 48 bit address space which is not
> a theoretical  limitation.  However, at least on linux you can map in
> and map out pretty quick (10 microseconds paired on my linux vm) so
> that's not so big of a deal.  Dealing with rapidly growing files is a
> problem.  That said, probably you are not going to want to reserve
> multiple gigabytes in 8k non contiguous chunks.
> 
> > But it's worse than that. Since you're not responsible for flushing
> > blocks to disk any longer you need some way to *unlock* a block when
> > it's possible to be flushed. That means when you flush the xlog you
> > have to somehow find all the blocks that might no longer need to be
> > locked and atomically unlock them. That would require new
> > infrastructure we don't have though it might not be too hard.
> > 
> > What would be nice is a mlock_until() where you eventually issue a
> > call to tell the kernel what point in time you've reached and it
> > unlocks everything older than that time.
Sorry for curious, but I think mlock is for swap prevent not for flush 
prevent.

> I wonder if there is any reason to mlock at all...if you are going to
> 'do' mmap, can't you just hide under current lock architecture for
> actual locking and do direct memory access without mlock?
> 
> merlin

mmap man do not say anything about when flush occurs when mmap is file and is 
shared, so flushes may be intended or not. Much more, this what I read, SysV 
shared memory is emulated by mmap (and I think this mmap is on /dev/shm)

Radek