I had a new look at this thread today, trying to figure out where we
are. I'm a bit confused.
One thing we have established: mmap()ing WAL files performs worse than
the current method, if pg_wal is not on a persistent memory device. This
is because the kernel faults in existing content of each page, even
though we're overwriting everything.
That's unfortunate. I was hoping that mmap() would be a good option even
without persistent memory hardware. I wish we could tell the kernel to
zero the pages instead of reading them from the file. Maybe clear the
file with ftruncate() before mmapping it?
That should not be problem with a real persistent memory device, however
(or when emulating it with DRAM). With DAX, the storage is memory-mapped
directly and there is no page cache, and no pre-faulting.
Because of that, I'm baffled by what the
v4-0002-Non-volatile-WAL-buffer.patch does. If I understand it
correctly, it puts the WAL buffers in a separate file, which is stored
on the NVRAM. Why? I realize that this is just a Proof of Concept, but
I'm very much not interested in anything that requires the DBA to manage
a second WAL location. Did you test the mmap() patches with persistent
memory hardware? Did you compare that with the pmem patchset, on the
same hardware? If there's a meaningful performance difference between
the two, what's causing it?
- Heikki