Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes

Поиск
Список
Период
Сортировка
От Dilip Kumar
Тема Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes
Дата
Msg-id CAFiTN-t9epn5M1sDEny6tcqhtXtd6QS48aPN9Z4-NcpXoFJOFw@mail.gmail.com
обсуждение исходный текст
Ответ на Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes  (SATYANARAYANA NARLAPURAM <satyanarlapuram@gmail.com>)
Ответы Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes  (Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>)
Список pgsql-hackers
On Thu, Dec 30, 2021 at 12:36 PM SATYANARAYANA NARLAPURAM <satyanarlapuram@gmail.com> wrote:

Yeah, I think that would make sense, even though we will be allowing a new backend to get connected insert WAL, and get committed but after that, it will be throttled.  However, if the number of max connections will be very high then even after we detected a lag there a significant amount WAL could be generated, even if we keep long-running transactions aside.  But I think still it will serve the purpose of what Satya is trying to achieve.

I am afraid there are problems with making the RPO check post releasing the locks. By this time the transaction is committed and visible to the other backends (ProcArrayEndTransaction is already called) though the intention is to block committing transactions that violate the defined RPO. Even though we block existing connections starting a new transaction, it is possible to do writes by opening a new connection / canceling the query. I am not too much worried about the lock contention as the system is already hosed because of the policy. This behavior is very similar to what happens when the Sync standby is not responding. Thoughts?

Yeah, that's true, but even if we are blocking the transactions from committing then also it is possible that a new connection can come and generate more WAL,  yeah but I agree with the other part that if you throttle after committing then the user can cancel the queries and generate more WAL from those sessions as well.  But that is an extreme case where application developers want to bypass the throttling and want to generate more WALs.   

--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com

В списке pgsql-hackers по дате отправления:

Предыдущее
От: Dilip Kumar
Дата:
Сообщение: Re: Per-table storage parameters for TableAM/IndexAM extensions
Следующее
От: Bharath Rupireddy
Дата:
Сообщение: Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes