On 01/29/2018 02:49 PM, Simon Riggs wrote:
> On 29 January 2018 at 13:34, Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:
>
>> The important detail is that we only really care
>> about aborts in transactions that modified catalogs in some way (e.g. by
>> doing DDL). But we can safely decode (and stream) changes up to the
>> point when the catalogs get modified, so we can do two different things
>> at that point:
>
> Safely? Hmm.
>
> I think what we are missing here is a test case, or a detailed
> description of what we think happens. I can't see why a concurrent
> abort would be unsafe, as in, it would make decoding explode.
>
> If all it does is give the wrong answer when we process DDL, then
> all we have to do is check for abort before and after we process any
> DDL during decoding. Handling DDL is rare, so that test won't cost
> much in terms of handling DDL. So ISTM that we can continue decoding
> after we hit DDL, we just need some care.
>
We do not decode DDL, but maybe I misunderstood what you mean.
The issue is that we consult catalogs during the decoding for various
reasons (to interpret table structure, see if the table is part of a
replication set, and the plugin is free to use arbitrary user catalogs).
Hitting the issue is tricky, because of timing issues.
But imagine a transaction like this.
BEGIN;
INSERT INTO t(a,b) VALUES (1,2);
SAVEPOINT s;
ALTER TABLE t ADD COLUMN c INT;
INSERT INTO t(a,b,c) VALUES (1,2,3);
ROLLBACK TO s;
COMMIT;
Now assume that we start streaming the transaction, and it happens like
this (T refers to the steps of the transaction):
1) start decoding the subtransaction (not yet aborted)
2) T: ROLLBACK TO s;
3) VACUUM the catalogs, which may remove pg_attribute rows for "c" or
make them inaccessible through indexes, etc.
4) inspect the new row (which we still have in reorderbuffer)
5) Kabooom! The row has column "c" which we don't see in the catalog.
> (My earlier comments were based on the idea that 2PC patch would
> simply block aborts, which of course will not work for streaming,
> hence the difference)
>
I'm sorry, but I don't quite see why wouldn't that work? There are far
too many ideas flying around on this thread, so I'm probably missing
something. Can you elaborate?
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services