Re: pglogical_output - a general purpose logical decoding output plugin

Поиск
Список
Период
Сортировка
От Craig Ringer
Тема Re: pglogical_output - a general purpose logical decoding output plugin
Дата
Msg-id CAMsr+YF8mu63hnhA6uFes=Y7aAjNU8g0o2g2PuUyYR+T7VVXQQ@mail.gmail.com
обсуждение исходный текст
Ответ на pglogical_output - a general purpose logical decoding output plugin  (Craig Ringer <craig@2ndquadrant.com>)
Ответы Re: pglogical_output - a general purpose logical decoding output plugin
Список pgsql-hackers
Hi all

Here's an updated pglogical_output patch.

Selected changes since v1:

    - add json protocol output support
    - fix encoding comparisons to use parsed encoding not string name
    - import protocol documentation
    - significantly expand pg_regress tests
    - move pglogical_output_plhooks to a top-level contrib
    - remove 9.4 compatibility


Notably, it now has support for a basic json-format text-based protocol as well as the main binary protocol. Mostly for debugging, testing and diagnostics.

I've used the newly added json support in pglogical_output to significantly expand the pg_regress tests, since the json mode can support text-mode decoding in the SQL interface to logical decoding, where the lsn, xid, etc can be skipped.

I've removed the Python based test framework. Test coverage in-tree is reduced as a result, with no coverage of the details of the binary protocol, no coverage of walsender based use, etc. I'll have a look at whether it'll be practical to convert the tests to Perl code driving pg_recvlogical as a co-process but don't think evaluation should wait on this.

I've also removed the 9.4 backward compatibility code from the version submitted for 9.6.

Docs conversion to SGML is still pending/WIP.

I've been unable to find a reasonable way to send the startup message before raising an ERROR when there's an issue with parameter/protocol negotiation. Advice/suggestions appreciated. The main issue is that the setup callback can't write to the protocol stream since in walsender mode the walsender isn't ready for writes yet. (Otherwise we could just write a record with InvalidXLogRecPtr, etc). Delaying the startup msg to the first begin callback works, as is done now. But delaying an error would involve allowing the begin callback to complete, then ERRORing at the *next* callback, which could be anything. Very ugly for what should be an uncommon case. So for now ERRORs are immediate, even though that makes negotiation much more difficult. Ideas welcomed.

(Cc'd a few people who expressed interest)

Вложения

В списке pgsql-hackers по дате отправления:

Предыдущее
От: Robert Haas
Дата:
Сообщение: Re: eXtensible Transaction Manager API
Следующее
От: Fabien COELHO
Дата:
Сообщение: Re: checkpointer continuous flushing