Re: Logical replication and multimaster
| От | Craig Ringer |
|---|---|
| Тема | Re: Logical replication and multimaster |
| Дата | |
| Msg-id | CAMsr+YECkyXz-+Kr0S0EWoW+cZuB5MGXte-tdSFZBB9sNzLUjA@mail.gmail.com обсуждение исходный текст |
| Ответ на | Re: Logical replication and multimaster (konstantin knizhnik <k.knizhnik@postgrespro.ru>) |
| Ответы |
Re: Logical replication and multimaster
("Shulgin, Oleksandr" <oleksandr.shulgin@zalando.de>)
Re: Logical replication and multimaster (konstantin knizhnik <k.knizhnik@postgrespro.ru>) |
| Список | pgsql-hackers |
On 3 December 2015 at 14:54, konstantin knizhnik <k.knizhnik@postgrespro.ru> wrote:
I'd really like to collaborate using pglogical_output if at all possible. Petr's working really hard to get the pglogical downstrem out too, with me helping where I can.And where I can get pglogical_output plugin? Sorry, but I can't quickly find reference with Google...
It's been submitted to this CF.
Any tests and comments would be greatly appreciated.
I have a version compatible with 9.4 and older in a separate tree I want to make public. I'll get back to you on that later today. It's the same code with a few more ifdefs and an uglier structure for the example hooks module (because it can be a separate contrib)¸so it's not that exciting.
You should be able to just "git remote add" that repo, "git fetch" and "git merge dev/pglogical-output" into your working tree.
Also I wonder if this plugin perform DDL replication (most likely not).
No, it doesn't. The way it's done in BDR is too intrusive and has to be reworked before it can be made more generally re-usable.
How I envision DDL replication working for pglogical (or anything else) is to take the DDL hooks added in 9.5 and use them with a separate DDL deparse extension based on Álvaro's deparse work. If you want to replicate DDL you make sure this extension is loaded then use it from your event triggers to capture DDL in a useful form and write it to a queue table where your downstream client can find it and consume it. That way the deparse code doesn't have to be embedded in the Pg backend like it is in BDR, and instead can be a reusable extension.
But then naive question - why DDL was excluded from logical replication protocol?
logical decoding can't handle DDL because all it sees is the effects of that DDL in the xlog as a series of changes to catalog tables, relfilenode changes, etc. It can't turn that back into the original DDL in any kind of reliable way. A downstream can't do very much with "rename relfilenode 1231 to 1241".
There are a few cases we might want to handle through decoding - in particular I'd like to be able to decode changes to rows in shared catalogs like pg_authid, since we can't handle that with DDL deparse. For things like DROP TABLE, CREATE TABLE, etc we really need DDL hooks. At least as I currently understand things.
So we try to capture DDL at a higher level. That's why event triggers were added (http://www.postgresql.org/docs/current/static/event-triggers.html) and why DDL deparse was implemented (https://commitfest-old.postgresql.org/action/patch_view?id=1610).
You can't just capture the raw DDL statement since there are issues with search_path normalization, etc. Similar problems to statement based replication exist. Deparse is required to get the DDL after it's converted to a utility statement so we can obtain it in an unambiguous form.
I'll add some explanation in pglogical_output's DESIGN.md for why DDL is not currently handled.
BTW, TRUNCATE _is_ handled by the way. In pglogical we use regular TRUNCATE triggers (marked tgisinternal) for that. There are some significant complexities around foreign keys, sequence reset, etc, which are not fully handled yet.
Are there some principle problems with it? In BDR it was handled in alternative way, using executor callback. It will be much easier if DDL can be replicated in the same way as normal SQL statements.
It can't. I wish it could.
В списке pgsql-hackers по дате отправления: