Re: Stream consistent snapshot via a logical decoding plugin as a series of INSERTs

Поиск
Список
Период
Сортировка
От Shulgin, Oleksandr
Тема Re: Stream consistent snapshot via a logical decoding plugin as a series of INSERTs
Дата
Msg-id CACACo5TLD_64JSQiF_8=r968xQrKhjMUVKQ6=-Em3Y=VCakdFA@mail.gmail.com
обсуждение исходный текст
Ответ на Re: Stream consistent snapshot via a logical decoding plugin as a series of INSERTs  ("Shulgin, Oleksandr" <oleksandr.shulgin@zalando.de>)
Список pgsql-hackers
Дерево обсуждения
On Fri, Jan 15, 2016 at 5:31 PM, Shulgin, Oleksandr <oleksandr.shulgin@zalando.de> wrote:

POC patch attached.  Findings:

1) Needs an actual slot for all the decode machinery to work (code depends on MyReplicationSlot being set).
2) Requires a core patch.
3) Currently only supports textual output, adding binary is trivial.


Acquiring a slot means this cannot be run in parallel from multiple backends.  Any ideas on how to overcome this (except for opening multiple slots with the same LSN)?
To obtain a consistent snapshot, the client still needs to take care of preserving and setting transaction snapshot properly.

Testing revealed a number of problems with memory handling in this code, a corrected v2 is attached.

Completely another problem is proper handling of SPI stack and releasing the replication slot.  The latter I'd like to avoid dealing with, because at the moment it's not possible to stream a number of relations in parallel using this POC function, so I'd rather move in a direction of not acquiring the replication slot at all.

The SPI problem manifests itself if I place a LIMIT on top of the query:

# SELECT pg_logical_slot_stream_relation('slot1', 'pg_catalog', 'pg_class') LIMIT 5;
WARNING:  relcache reference leak: relation "pg_class" not closed
WARNING:  transaction left non-empty SPI stack
HINT:  Check for missing "SPI_finish" calls.

I wonder if there is a way to install some sort of cleanup handler that will be called by executor?

--
Alex

Вложения

В списке pgsql-hackers по дате отправления:

Предыдущее
От: "Igal @ Lucee.org"
Дата:
Сообщение: Re: make error - libpqdll.def No such file or directory
Следующее
От: Alvaro Herrera
Дата:
Сообщение: Re: make error - libpqdll.def No such file or directory