Re: [HACKERS] Built-in plugin for logical decoding output

On 24/09/17 02:41, Euler Taveira wrote:
> 2017-09-23 14:01 GMT-03:00 Alvaro Hernandez <aht@ongres.com>:
>>      However, AFAIK, AWS's DMS uses it for production purposes (see
>> http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_PostgreSQL.html).
>>
> It seems a bad idea. AFAICS test_decoding was not designed to be a
> ready-for-production plugin. It is just a proof of concept for logical
> decoding.
    Yes, this is what I heard and read.
    However, if DMS uses it for what I'd call production use, I assume 
it is actually production quality. I bet they do enough testing, and 
don't ship software to potentially millions of customers if it doesn't 
work well. So... first, I'd consider this a a sign of robustness. 
Second..... my hats off for the plugin code ;)

>>      I would be happy to see another logical decoding plugin into core
>> starting on 11. However, this also poses a bit of a challenge for middleware
>> implementors: you need to support one for 9.4-9.5 (test_decoding), another
>> for 10 (pgoutput) and maybe another for 11 onwards. The idea of asking users
>> to install a binary plugin is very unsexy, so these are the options
>> available.
>>
> wal2json works for 9.4+ (besides the WAL messages I committed a month
> ago). Since this boat was already shipped we can arrange some packages
> for 9.4-10 (an external project) and ask vendors to support the
> backward-compatible plugin. The middleware implementor will have to
> support this new plugin format. Being JSON a widespread format, it is
> easier to refactor the code to parse JSON.
    I agree its far better to parse JSON than the test_decoding output. 
But asking any potential user to install a dynamic library, from a third 
party website, which will need to be compiled for many potential 
OSes/Archs, or even impossible if running on a managed environment... is 
not a great experience. Unless PostgreSQL would backport a plugin and 
ship it in newer releases, if test_decoding is good enough, I'd rather 
stick to it.

>
>>      However, having said that, and while json is a great output format for
>> interoperability, if there's a discussion on which plugin to include next,
>> I'd also favor one that has some more compact representation format (or that
>> supports several formats, not only json).
>>
> We could certainly extend pgoutput to support more than one format
> (like pglogical did AFAIR), however, we wouldn't reuse code (different
> formats) and will have a fat plugin (I don't foresee a plugin using
> different formats in the same connection. It is difficult to
> coordinate a change like that having only one-way communication).
>
    I think pgoutput is what it is. Maybe instead than growing it, my 
+1 would be to add a new plugin that rather than being json only, would 
also support other formats, like an efficient binary serialization.

    Álvaro


-- 

Alvaro Hernandez


-----------
OnGres



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers