Re: [HACKERS] Detecting schema changes during logical replication

Поиск
Список
Период
Сортировка
От Craig Ringer
Тема Re: [HACKERS] Detecting schema changes during logical replication
Дата
Msg-id CAMsr+YE3OhgDHbMg4OnumpKHajbokiEkJ5hWQkx3z+Lj12PHeQ@mail.gmail.com
обсуждение исходный текст
Ответ на Re: [HACKERS] Detecting schema changes during logical replication  (Daniele Varrazzo <daniele.varrazzo@gmail.com>)
Ответы Re: [HACKERS] Detecting schema changes during logical replication
Список pgsql-hackers


On 8 May 2017 05:56, "Daniele Varrazzo" <daniele.varrazzo@gmail.com> wrote:
On Sun, May 7, 2017 at 8:04 PM, Andres Freund <andres@anarazel.de> wrote:
> Hi,
>
> On 2017-05-07 19:27:08 +0100, Daniele Varrazzo wrote:
>> I'm putting together a replication system based on logical
>> replication.
>
> Interesting.  If you very briefly could recap what it's about... ;)

I need to replicate some tables from a central database into the
database that should run a secondary system. For a similar use case we
have used Londiste in the past, which has served us good, but its
usage has not been problem-free. Logical decoding seems much less
invasive on the source database than a trigger-based replication
solution, and has less moving part to care about and maintain.

For the moment I'm hacking into a fork of Euler project for wal
decoding into json (https://github.com/dvarrazzo/wal2json), mostly
adding configurability, so that we may be able to replicate only the
tables we need, skip certain fields etc. I'm also taking a look at
minimising the amount of information produced: sending over and over
the column names and types for every record seems a waste, hence my
question.

Sounds like you're reimplementing pglogical (http://2ndquadrant.com/pglogical) on top of a json protocol.

Pglogical is open source and hackable to meet your needs. We're also happy to accept patches with appropriate design discussion and code review to make sure they will aid or not hinder other users and won't add undue maintenance burden.

Rather than repeat the same work, maybe it's worth picking it up as a starting point. It's the extension-based progenitor of the in-core logical rep code, but portable from 9.4 to 9.6 (and soon 10) with a bunch of features that didn't make it into Pg's version yet.

I have no reason to object to your doing it yourself, and you're welcome to use pglogical as a reference for how to do things (see the license). It just seems like a waste.

В списке pgsql-hackers по дате отправления:

Предыдущее
От: Michael Paquier
Дата:
Сообщение: Re: [HACKERS] Concurrent ALTER SEQUENCE RESTART Regression
Следующее
От: Masahiko Sawada
Дата:
Сообщение: Re: [HACKERS] PG 10 release notes