Memory mapping (Was: Safe/Fast I/O ...)

While having some spare two hours I was just looking at the current code
of postgres. I was trying to estimate how would it fit to the current
postgres guts.

Finally I've found more proofs that memory mapping would do a lot to
current performance, but I must admit that current storage manager is
pretty read/write oriented. It would be easier to integrate memory
mapping into buffer manager. Actually buffer manager role is to map some
parts of files into memory buffers. However it takes a lot to get
through several layers (smgr and finally md).

I noticed that one of the very important features of mmaping is that you
can sync the buffer (even some part of it), not the whole file. So if
there would be some kind of page level locking, it would be absolutly
necessary to make sure that only committed pages are synced and we don't
overload the IO with unfinished things.

Also, I think that there is no need to create buffers in shared memory.
I have just tested that if you map files with MAP_SHARED attribute set,
then each proces is working on exactly the same copy of memory.

I have also noticed more interesting things, maybe somebody would
clarify on that since I'm not so literate with mmaping. First thing I
was wondering about was how would we deal with open descriptor limits if
we use direct buffer-to-file mappings. While currently buffers are
isolated from files it's possible to close some descriptors without
throwing buffers. However it seems (tried it) that memory mapping works
even after a file descriptor is closed. So, is this possible to cross
the limit of open files by using memory mapping? Or maybe the descriptor
remains open until munmap call? Or maybe it's just a Linux feature?

Mike

--
WWW: http://www.lodz.pdi.net/~mimo  tel: Int. Acc. Code + 48 42 148340
add: Michal Mosiewicz  *  Bugaj 66 m.54 *  95-200 Pabianice  *  POLAND