Re: Stopping logical replication protocol

Поиск
Список
Период
Сортировка
От Craig Ringer
Тема Re: Stopping logical replication protocol
Дата
Msg-id CAMsr+YFyPUoy8k0Q1pc1B-Npx7fkw3pVm+GQ43xbDfOxA77rNQ@mail.gmail.com
обсуждение исходный текст
Ответ на Re: Stopping logical replication protocol  (Vladimir Gordiychuk <folyga@gmail.com>)
Ответы Re: Stopping logical replication protocol  (Vladimir Gordiychuk <folyga@gmail.com>)
Список pgsql-hackers
On 10 May 2016 at 19:41, Vladimir Gordiychuk <folyga@gmail.com> wrote:

Fair enough. Though I don't understand why you'd be doing this often enough that you'd care about reopening connections. What is the problem you are trying to solve with this? The underlying reason you need this change?

First reason it clear API in pgjdc. Second reason it ability fast enough rollack to one of the previous LSN and repeat it. My use case for second reason - I have logical decoding extension that prepare only data as key-value pair without information about (insert, update, delete) for example if it delete as key I use primary key for table and as value null. Via pgjdc by replication protocol connects receiver data, consumer group changes to batch and send it to Kafka. If some problem occurs during delivery to kafka consumer, I should stop current replication, go back to success LSN and repeat all messages from success LSN. I think it operation can be quite common, but reopen connection or not stopping replication will increase latency.

It will, but not tons. The biggest cost (at least if there are any long running xacts) will be replaying since the restart_lsn when you restart the decoding session, which will happen whether you reconnect or just stop decoding and return to command mode.

 
Anyway, here's a draft patch that does the parts other than the reorder buffer processing stop. It passes 'make check' and the src/test/recovery tests, but I haven't written anything to verify the client-initiated abort handling. You have test code for this and I'd be interested to see the results.

What about keepAlive package when we already send/receive CopyDone? Is It really necessary?

No, I don't think it is, and I applied the change you made to suppress it.
 
I think it's worth making the next step, where you allow reorder buffer commit processing to be interrupted, into a separate patch on top of this one. They're two separate changes IMO.

We will continue in the current thread, or new? I interesting in both patch for my solution and pgjbc driver. 

Same thread, I just think these are two somewhat separate changes. One is just in the walsender and allows return to command mode during waiting for WAL. The other is more intrusive into the reorder buffer etc and allows aborting decoding during commit processing. So two separate patches make sense here IMO, one on top of the other.

I use git and just "git format-patch -2" to prepare a stack of two patches from two separate commits.

--
 Craig Ringer                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

В списке pgsql-hackers по дате отправления:

Предыдущее
От: Amit Kapila
Дата:
Сообщение: Hash Indexes
Следующее
От: Robert Haas
Дата:
Сообщение: Re: Patch for German translation