Re: [PATCH 10/16] Introduce the concept that wal has a 'origin' node

On 21 June 2012 02:32, Heikki Linnakangas
<heikki.linnakangas@enterprisedb.com> wrote:
> On 20.06.2012 17:35, Simon Riggs wrote:
>>
>> On 20 June 2012 16:23, Heikki Linnakangas
>> <heikki.linnakangas@enterprisedb.com>  wrote:
>>>
>>> On 20.06.2012 11:17, Simon Riggs wrote:
>>>>
>>>>
>>>> On 20 June 2012 15:45, Heikki Linnakangas
>>>> <heikki.linnakangas@enterprisedb.com>    wrote:
>>>>>
>>>>>
>>>>> So, if the origin id is not sufficient for some conflict resolution
>>>>> mechanisms, what extra information do you need for those, and where do
>>>>> you put it?
>>>>
>>>>
>>>> As explained elsewhere, wal_level = logical (or similar) would be used
>>>> to provide any additional logical information required.
>>>>
>>>> Update and Delete WAL records already need to be different in that
>>>> mode, so additional info would be placed there, if there were any.
>>>>
>>>> In the case of reflexive updates you raised, a typical response in
>>>> other DBMS would be to represent the query
>>>>   UPDATE SET counter = counter + 1
>>>> by sending just the "+1" part, not the current value of counter, as
>>>> would be the case with the non-reflexive update
>>>>   UPDATE SET counter = 1
>>>>
>>>> Handling such things in Postgres would require some subtlety, which
>>>> would not be resolved in first release but is pretty certain not to
>>>> require any changes to the WAL record header as a way of resolving it.
>>>> Having already thought about it, I'd estimate that is a very long
>>>> discussion and not relevant to the OT, but if you wish to have it
>>>> here, I won't stop you.
>>>
>>>
>>>
>>> Yeah, I'd like to hear briefly how you would handle that without any
>>> further
>>> changes to the WAL record header.
>>
>>
>> I already did:
>>
>>>> Update and Delete WAL records already need to be different in that
>>>> mode, so additional info would be placed there, if there were any.
>>
>>
>> The case you mentioned relates to UPDATEs only, so I would suggest
>> that we add that information to a new form of update record only.
>>
>> That has nothing to do with the WAL record header.
>
>
> Hmm, so you need the origin id in the WAL record header to do filtering.
> Except when that's not enough, you add some more fields to heap update and
> delete records.

Yes

> Don't you think it would be simpler to only add the extra fields to heap
> insert, update and delete records, and leave the WAL record header alone? Do
> you ever need extra information on other record types?

No extra info on other record types, in general at least. Doing it
that way is the most logical place, just not the most efficient.

> The other question is, *what* information do you need to put there? We don't
> necessarily need to have a detailed design of that right now, but it seems
> quite clear to me that we need more flexibility there than this patch
> provides, to support more complicated conflict resolution.

Another patch already covers that for the common non-reflexive case.
More on conflict resolution soon, I guess. I was going to begin
working on that in July. Starting with the design discussed on list,
of course. This patch has thrown up stuff I thought was
compartmentalised, but I was wrong.

> I'm not saying that we need to implement all possible conflict resolution
> algorithms right now - on the contrary I think conflict resolution belongs
> outside core

It's a pretty standard requirement to have user defined conflict
handling, if that's what you mean.

I'm OK with conflict handling being outside of core as a module, if
that's what people think. We just need a commit hook. That is more
likely to allow a workable solution with 9.3 as well, ISTM. It's
likely to be contentious...

> - but if we're going to change the WAL record format to support
> such conflict resolution, we better make sure the foundation we provide for
> it is solid.

Agreed

> BTW, one way to work around the lack of origin id in the WAL record header
> is to just add an origin-id column to the table, indicating the last node
> that updated the row. That would be a kludge, but I thought I'd mention it..

err, I hope you mean that to be funny. (It wouldn't actually work either)

--
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services