On 28 October 2014 14:53, Robert Haas <robertmhaas@gmail.com> wrote:
> On Tue, Oct 28, 2014 at 10:22 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
>> Or put it another way, it will be easier to write new index AMs
>> because we'll be able to skip the WAL part until we know we want it.
>
> I like the feature you are proposing, but I don't think that we should
> block Alexander from moving forward with a more-extensible WAL format.
> I believe that's a general need even if we get the features you're
> proposing, which would reduce the need for it. After all, if somebody
> builds an out-of-core index AM, ignoring WAL-logging, and then decides
> that it works well enough that they want to add WAL-logging, I think
> we should make that possible without requiring them to move the whole
> thing in-core.
I'm not proposing an alternate or additional feature.
I'm saying that the first essential step in adding WAL support to new
AMs is to realise that they *will* have bugs (since with the greatest
respect, the last two AMs from our friends did have multiple bugs) and
so we must have a mechanism that prevents such bugs from screwing
everything else up. Which is the mark-corrupt-index and rebuild
requirement.
We skip straight to the add-buggy-AMs part at our extreme peril. We've
got about 10x as many users now since the 8.x bugs and all the new
users like the reputation Postgres has for resilience. I think we
should put the safety net in place first before we start to climb.
The patch as submitted doesn't have any safety checks for whether the
WAL records refer to persistent objects defined by the AM. At the very
least we need to be able to isolate an AM to only screw up their own
objects. Such checks look like they'd require some careful thought and
refactoring first.
-- Simon Riggs http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services