Re: LWLock Queue Jumping

Yes, good idea.  But thinking about it as a patch to be applied, rather than a proof of concept, I think the best solution would be to add a third argument (boolean Priority) to LWLockAquire, and hunt down all existing invocations and change them to include false as the extra argument.  Copying 160 lines of code to change 4 of them in the copy is temporarily easier, but not a good solution for the long term.

I'd have a hard time coming up with the full origianl patch, as my changes for files other than lwlock.c were blown away by parallel efforts and an rsync to the repo.  The above was just an exploratory tool, not proposed as an actual patch to be applied to HEAD.  If we want to add a parameter to the existing LWLockAcquire, I'll work on coming up with a tested patch for that.  My testing was to run the normal regression test (which often failed to detect my early buggy implementations), then load testing with pgbench (which always (that I know of) found them when -c > 1) and a custom Perl script I use.  Since WALWriteLock is heavily used and contended under pgbench -c 20, and lwlock is agnostic to the exact identity of the underlying lock, I think this test was pretty thorough for the implementation.  But not of course for starvation issues, which would have to be tested on a case by case basis when a specific Acquire invocation is changed to be high priority.

If you have ideas for other tests to do, or corner cases that are likely to be overlooked by my tests, I'll try to work tests for them in too.
 

I had not mentioned it before.  The change helped by ~50% or so when wal_buffers was undersized (kept at the default setting) but did not help significantly when wal_buffers was generously sized.  I didn't think we would be interested in introducing a new locking procedure just to optimize performance for a poorly configured server.  But if we are to introduce this method for other reasons, I think it should be used for AdvanceXLInsertBuffer as well.
 
Cheers,

Jeff