Re: [PATCH 14/16] Add module to apply changes from an apply-cache using low-level functions

On Wednesday, June 13, 2012 08:50:42 PM Christopher Browne wrote:
> On Wed, Jun 13, 2012 at 7:28 AM, Andres Freund <andres@2ndquadrant.com> 
wrote:
> > From: Andres Freund <andres@anarazel.de>
> > 
> > We decided to use low level functions to do the apply instead of
> > producing sql statements containing the data (or using prepared
> > statements) because both, the text conversion and the full executor
> > overhead aren't introduce a significant overhead which is unneccesary if
> > youre using the same version of pg on the same architecture.
> > 
> > There are loads of use cases though that require different methods of
> > applyin though - so the part doing the applying from an ApplyCache is
> > just a bunch of well abstracted callbacks getting passed all the
> > required knowledge to change the data representation into other formats.
> 
> It's important to make sure that it's not going to be *too* difficult
> to "jump through the hoops" necessary to apply changes on a different
> version.
I aggree. But I don't see it as a feature of the first version for the moment. 
Getting a base set of features into 9.3 is going to be hard enough as-is. But 
I think there is enough interest from all sides to make that possible cross-
version.

> While pg_upgrade has diminished the need to use replication to handle
> cross-version/architecture upgrades, I don't think it has brought that
> to zero.
Aggreed.

> One other complication I'll observe...  The assumption is being made
> that UPDATE/DELETE will be handled via The Primary Key.  For the most
> part, I approve of this.  Once upon a time, Slony had a misfeature
> where you could tell it to add in a surrogate primary key, and that
> caused no end of trouble.  However, the alternative, that *does* seem
> to work alright, is to allow selecting a candidate primary key, that
> is, a set of columns that have UNIQUE + NOT NULL characteristics.  I
> could see people having a problem switching over to use this system if
> they MUST begin with a 'Right Proper Primary Key.'  If they start with
> a system with a 2TB table full of data that lacks that particular
> constraint, that could render them unable to use the facility.
It wouldn't need that much code to allow candidate keys. The data 
representation in the catalogs is a bit unfriendly for that, but there has 
been talk about changing that for some time now. I am not convinced that its 
worth the cpu cycles though.

Btw, you can convert a unique key to a primary key since 9.1. The unique key 
previously can be created CONCURRENTLY.

> > Missing:
> > 
> > - TOAST handling. For physical apply not much needs to be done because
> > the toast inserts will have been made beforehand. There needs to be an
> > option in ApplyCache that helps reassembling TOAST datums to make it
> > easier to write apply modules which convert to text.
> Dumb question: Is it possible that two nodes would have a different
> idea as to which columns need to get toasted?  I should think it
> possible for nodes to be configured with different values for TOAST
> policies, and while it's likely pretty dumb to set them to have
> different handlings, it would seem undesirable to not bother looking,
> and find the backend crashing due to an un-noticed mismatch.
I don't think it should be possible to configure the toast configurations 
differently if you use the "binary apply" mode. But even if it were a value 
which is toasted although the local policy says it should not be wouldn't 
cause any problems as far as I can see.
The one problem that could cause problems for that are different page sizes et 
al, but that needs to be prohibited anyway.

Andres

-- Andres Freund                       http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services