Excerpts from Robert Haas's message of mié jun 08 23:24:34 -0400 2011:
> I did a brief literature search for papers on breaking the
> WAL-serialization bottleneck today and hit upon this:
>
> Aether: A Scalable Approach to Logging, Ryan Johnson, Ippokratis Pandis, et al.
> http://infoscience.epfl.ch/record/149436/files/vldb10aether.pdf
Great.
> 1. Subdivide XLOG insertion into three operations: (1) allocate space
> in the log buffer, (2) copy the log records into the allocated space,
> and (3) release the space to the buffer manager for eventual write to
> disk. AIUI, WALInsertLock currently covers all three phases of this
> operation, but phase 2 can proceed in parallel. It's pretty easy to
> imagine maintain one pointer that references the next available byte
> of log space (let's call this the "next insert" pointer), and a second
> pointer that references the byte following the last byte known to be
> written (let's call this the "insert done" pointer).
I think this can be done more simply if instead of a single "insert
done" pointer you have an array of them, one per backend; there's also a
global pointer that can be advanced per the minimum of the bunch, which
you can calculate with some quick locking of the array. You don't need
to sleep at all, except to update the array and calculate the global
ptr, so this is probably also faster.
> Second, they aren't really using locks, unless you count
> bus locks - they appear to have implemented most or all of it via
> CAS-based lock-free algorithms, which is probably well-justified
> optimization effort.
Oh, hmm ...
--
Álvaro Herrera <alvherre@commandprompt.com>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support