Re: row filtering for logical replication

Поиск
Список
Период
Сортировка
Искать
От
Euler Taveira
Тема
Re: row filtering for logical replication
Дата
Msg-id
d65b0801-34bb-4db2-b0d4-8e39852c9f9d@www.fastmail.com
Ответ на
Список
Дерево обсуждения
row filtering for logical replication Euler Taveira <euler@timbira.com.br>
Re: row filtering for logical replication David Fetter <david@fetter.org>
Re: row filtering for logical replication Euler Taveira <euler@timbira.com.br>
Re: row filtering for logical replication David Fetter <david@fetter.org>
Re: row filtering for logical replication Erik Rijkers <er@xs4all.nl>
Re: row filtering for logical replication Andres Freund <andres@anarazel.de>
Re: row filtering for logical replication David Steele <david@pgmasters.net>
Re: row filtering for logical replication Euler Taveira <euler@timbira.com.br>
Re: row filtering for logical replication David Steele <david@pgmasters.net>
Re: row filtering for logical replication Michael Paquier <michael@paquier.xyz>
Re: row filtering for logical replication Erik Rijkers <er@xs4all.nl>
Re: row filtering for logical replication Euler Taveira <euler@timbira.com.br>
Re: row filtering for logical replication Euler Taveira <euler@timbira.com.br>
Re: row filtering for logical replication Erik Rijkers <er@xs4all.nl>
Re: row filtering for logical replication Erik Rijkers <er@xs4all.nl>
Re: row filtering for logical replication Euler Taveira <euler@timbira.com.br>
Re: row filtering for logical replication Erik Rijkers <er@xs4all.nl>
Re: row filtering for logical replication Petr Jelinek <petr.jelinek@2ndquadrant.com>
Re: row filtering for logical replication David Fetter <david@fetter.org>
Re: row filtering for logical replication Petr Jelinek <petr.jelinek@2ndquadrant.com>
Re: row filtering for logical replication Euler Taveira <euler@timbira.com.br>
Re: row filtering for logical replication Alvaro Herrera <alvherre@2ndquadrant.com>
Re: row filtering for logical replication Andres Freund <andres@anarazel.de>
Re: row filtering for logical replication a.kondratov@postgrespro.ru
Re: row filtering for logical replication Euler Taveira <euler@timbira.com.br>
Re: row filtering for logical replication Alexey Zagarin <zagarin@gmail.com>
Re: row filtering for logical replication Euler Taveira <euler@timbira.com.br>
Re: row filtering for logical replication Erik Rijkers <er@xs4all.nl>
Re: row filtering for logical replication Euler Taveira <euler@timbira.com.br>
Re: row filtering for logical replication Erik Rijkers <er@xs4all.nl>
Re: row filtering for logical replication Alexey Zagarin <zagarin@gmail.com>
Re: row filtering for logical replication Euler Taveira <euler@timbira.com.br>
Re: row filtering for logical replication Erik Rijkers <er@xs4all.nl>
Re: row filtering for logical replication Alexey Zagarin <zagarin@gmail.com>
Re: row filtering for logical replication Euler Taveira <euler@timbira.com.br>
Re: row filtering for logical replication movead li <movead.li@highgo.ca>
Re: row filtering for logical replication Euler Taveira <euler@timbira.com.br>
Re: row filtering for logical replication Euler Taveira <euler@timbira.com.br>
Re: row filtering for logical replication Amit Langote <amitlangote09@gmail.com>
Re: row filtering for logical replication Amit Langote <amitlangote09@gmail.com>
Re: row filtering for logical replication Michael Paquier <michael@paquier.xyz>
Re: row filtering for logical replication Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: row filtering for logical replication Euler Taveira <euler@timbira.com.br>
Re: row filtering for logical replication Craig Ringer <craig@2ndquadrant.com>
Re: row filtering for logical replication David Steele <david@pgmasters.net>
Re: row filtering for logical replication David Steele <david@pgmasters.net>
Re: row filtering for logical replication "Euler Taveira" <euler@eulerto.com>
Re: row filtering for logical replication japin <japinli@hotmail.com>
Re: row filtering for logical replication "Euler Taveira" <euler@eulerto.com>
Re: row filtering for logical replication japin <japinli@hotmail.com>
Re: row filtering for logical replication Michael Paquier <michael@paquier.xyz>
Re: row filtering for logical replication japin <japinli@hotmail.com>
Re: row filtering for logical replication japin <japinli@hotmail.com>
Re: row filtering for logical replication "Euler Taveira" <euler@eulerto.com>
Re: row filtering for logical replication Önder Kalacı <onderkalaci@gmail.com>
Re: row filtering for logical replication Andres Freund <andres@anarazel.de>
Re: row filtering for logical replication Masahiko Sawada <sawada.mshk@gmail.com>
Re: row filtering for logical replication Önder Kalacı <onderkalaci@gmail.com>
Re: row filtering for logical replication Petr Jelinek <petr.jelinek@2ndquadrant.com>
Re: row filtering for logical replication Fabrízio de Royes Mello <fabriziomello@gmail.com>
Re: row filtering for logical replication Petr Jelinek <petr.jelinek@2ndquadrant.com>
Re: row filtering for logical replication Fabrízio de Royes Mello <fabriziomello@gmail.com>
Re: row filtering for logical replication Stephen Frost <sfrost@snowman.net>
Re: row filtering for logical replication Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: row filtering for logical replication Stephen Frost <sfrost@snowman.net>
Re: row filtering for logical replication Petr Jelinek <petr.jelinek@2ndquadrant.com>
Re: row filtering for logical replication Stephen Frost <sfrost@snowman.net>
Re: row filtering for logical replication Petr Jelinek <petr.jelinek@2ndquadrant.com>
Re: row filtering for logical replication Petr Jelinek <petr.jelinek@2ndquadrant.com>
Re: row filtering for logical replication Tomas Vondra <tomas.vondra@2ndquadrant.com>
Re: row filtering for logical replication Hironobu SUZUKI <hironobu@interdb.jp>
Re: row filtering for logical replication Craig Ringer <craig@2ndquadrant.com>
Re: row filtering for logical replication Euler Taveira <euler@timbira.com.br>
Re: row filtering for logical replication Stephen Frost <sfrost@snowman.net>
Re: row filtering for logical replication Petr Jelinek <petr.jelinek@2ndquadrant.com>
Re: row filtering for logical replication Euler Taveira <euler@timbira.com.br>
Re: row filtering for logical replication Stephen Frost <sfrost@snowman.net>
Re: row filtering for logical replication Petr Jelinek <petr.jelinek@2ndquadrant.com>
Re: row filtering for logical replication Stephen Frost <sfrost@snowman.net>
Re: row filtering for logical replication Petr Jelinek <petr.jelinek@2ndquadrant.com>
On Tue, Aug 24, 2021, at 4:46 AM, Peter Smith wrote:
I have used debug logging to confirm that what Amit wrote [1] is
correct; the row-filter ExprState of *every* table's row_filter will
be invalidated (and so subsequently gets rebuilt) when the user
changes the PUBLICATION tables. This was a side-effect of the
rel_sync_cache_publication_cb which is freeing the cached ExprState
and setting the entry->replicate_valid = false; for *every* entry.

So yes, the ExprCache is getting rebuilt for some situations where it
is not strictly necessary to do so.
I'm afraid we are overenginnering this feature. We already have a cache
mechanism that was suggested (that shows a small improvement). As you said the
gain for this new improvement is zero or minimal (it depends on your logical
replication setup/maintenance).

1. Although the ExprState cache is effective, in practice the
performance improvement was not very much. My previous results [2]
showed only about 2sec saving for 100K calls to the
pgoutput_row_filter function. So I think eliminating just one or two
unnecessary calls in the get_rel_sync_entry is going to make zero
observable difference.

2. IMO it is safe to expect that the ALTER PUBLICATION is a rare
operation relative to the number of times that pgoutput_row_filter
will be called (the pgoutput_row_filter is quite a "hot" function
since it is called for every INSERT/UPDATE/DELETE). It will be orders
of magnitude difference 1:1000, 1:100000 etc.

~~

Anyway, I have implemented the suggested cache change because I agree
it is probably theoretically superior, even if in practice there is
almost no difference.
I didn't inspect your patch carefully but it seems you add another List to
control this new cache mechanism. I don't like it. IMO if we can use the data
structures that we have now, let's implement your idea; otherwise, -1 for this
new micro optimization.

[By the way, it took some time to extract what you changed. Since we're trading
patches, I personally appreciate if you can send a patch on the top of the
current one. I have some changes too and it is time consuming incorporating
changes in the main patch.]


--
Euler Taveira

В списке pgsql-hackers по дате отправления
От: Michael Paquier
Дата:
От: Fujii Masao
Дата:
FAQ