Re: [HACKERS] Built-in plugin for logical decoding output
От | Alvaro Hernandez |
---|---|
Тема | Re: [HACKERS] Built-in plugin for logical decoding output |
Дата | |
Msg-id | fd4f57d2-5f7d-f79c-95d3-f0480c058dd7@ongres.com обсуждение исходный текст |
Ответ на | Re: [HACKERS] Built-in plugin for logical decoding output (Euler Taveira <euler@timbira.com.br>) |
Ответы |
Re: [HACKERS] Built-in plugin for logical decoding output
Re: [HACKERS] Built-in plugin for logical decoding output |
Список | pgsql-hackers |
On 23/09/17 18:42, Euler Taveira wrote: > 2017-09-22 19:28 GMT-03:00 Gregory Brail <gregbrail@google.com>: >> We have been working on a project that makes extensive use of logical >> replication for use inside Apigee (which is a very small part of Google): >> >> https://github.com/apigee-labs/transicator >> >> In order to do this, we had to write our own logical replication plugin >> because the supplied "test_decoding" plugin from the "contrib" directory was >> hard for us to work with. Primarily: >> > test_decoding is a proof of concept to illustrate the logical decoding > potential. It is not intended for production. However, AFAIK, AWS's DMS uses it for production purposes (see http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_PostgreSQL.html). > I developed wal2json [1] > for general use. It outputs changes in JSON. It was one of the first > logical decoding plugins. > >> 1) It doesn't include all the fields that we need for Transicator (most >> importantly we need the LSN and the 64-bit transaction ID). >> > wal2json includes both. > >> 2) It outputs a text format that is hard to parse. >> > There are a lot of JSON parsers. > >> I imagine that other users of logical decoding are facing similar problems. >> >> Would the community support the development of another plugin that is >> distributed as part of "contrib" that addresses these issues? I'd be happy >> to submit a patch, or GitHub repo, or whatever works best as an example. >> (Also, although Transicator uses protobuf, I'm happy to have it output a >> simple binary format as well.) >> > There was a prior discussion and it was suggestted that we have a > ready-for-production plugin in core (besides pgoutput). It was > suggested [1] that I submit wal2json for 11. I'm in process to clean > up the code and hope to submit it to CF2. I would be happy to see another logical decoding plugin into core starting on 11. However, this also poses a bit of a challenge for middleware implementors: you need to support one for 9.4-9.5 (test_decoding), another for 10 (pgoutput) and maybe another for 11 onwards. The idea of asking users to install a binary plugin is very unsexy, so these are the options available. However, having said that, and while json is a great output format for interoperability, if there's a discussion on which plugin to include next, I'd also favor one that has some more compact representation format (or that supports several formats, not only json). Regards, Álvaro -- Alvaro Hernandez ----------- OnGres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
В списке pgsql-hackers по дате отправления:
Предыдущее
От: Euler TaveiraДата:
Сообщение: Re: [HACKERS] Built-in plugin for logical decoding output