LWLockAcquire with priority

Thinking about how to reduce the effects of certain race conditions
makes me think about whether it is possible to make a function called
LWLockAcquireWithPriority().

We already allow "queue jumping" when lock mode != LW_EXCLUSIVE, so
queue jumping based upon an assigned priority rather than arrival time
might also work.

If we maintain two tails, rather than one, we can allow two entry points
onto the lock queue. One is a "fast track" entry point for hi priority
requests, while the other is the normal track for most requests.

I would propose we use this:
* on WALInsertLock requests for commit (not abort)
* on WALInsertLock and SLRU requests for ExtendCLog,
ExtendMultiXactOffset, ExtendMultiXactMember

This will prevent commits being stalled when we occasionally switch clog
and multixact pages, plus it also stops commits from being stalled when
there are heavy writers in progress.

If we use this only on certain locks, we can use macros to make
LWLockAcquire point to the new function without changes to most code.

So we have new function 
LWLockAcquireWithPriority(LWLockId lockid, LWLockMode mode,             bool hipriority)
and 
LWLockAcquireWithoutPriority(LWLockId lockid, LWLockMode mode)

LWLockAcquire() maps to LWLockAcquireWithoutPriority for all locks
except for WALInsertLock and SLRU locks, which map to
LWLockAcquireWithPriority(x, x, false)

LockAcquireWithPriority(x, x, true) would then be used in key places as
suggested above.

-- Simon Riggs           www.2ndQuadrant.comPostgreSQL Training, Services and Support