On Thu, May 3, 2012 at 5:59 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Robert Haas <robertmhaas@gmail.com> writes:
>> [ CLOG extension is horrid for concurrency ]
>
> Yeah. When that code was designed, a page's worth of transactions
> seemed like a lot so we didn't worry too much about performance glitches
> when we crossed a page boundary. It's time to do something about it
> though.
>
> The idea of extending CLOG in advance, so that the work doesn't have to
> be done with quite so many locks held, sounds like a plan to me. The
> one thing I'd worry about is that extension has to interact with
> freezing of very old XIDs and subsequent removal of old clog pages;
> make sure that pages will get removed before they could possibly
> get created again.
>
>> First, do we really need to WAL-log CLOG extension at all? Perhaps
>> recovery should simply extend CLOG when it hits a commit or abort
>> record that references a page that doesn't exist yet.
>
> Maybe, but see above. I'd be particularly worried about this in a hot
> standby situation, as you would then end up with HS queries seeing XIDs
> (in tuples) for which there was no clog page yet. I'm inclined to think
> it's better to continue to WAL-log it, but try to arrange to do that
> without holding the other locks that are now involved.
Why not switch to 1 WAL record per file, rather than 1 per page. (32
pages, IIRC).
We can then have the whole new file written as zeroes by a background
process, which needn't do that while holding the XidGenLock.
My earlier patch to do background flushing from bgwriter can be
extended to do that.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services