Re: viewing connectioninfo used by subscriber on the publicationserver when inactive

Поиск
Список
Период
Сортировка
От Keith Fiske
Тема Re: viewing connectioninfo used by subscriber on the publicationserver when inactive
Дата
Msg-id CAODZiv5NhMO3dDsDeBm1q6nVvwPnhTuRe7TE7ON3qi+Eha2nfA@mail.gmail.com
обсуждение исходный текст
Ответ на Re: viewing connectioninfo used by subscriber on the publicationserver when inactive  (Wim Bertels <wim.bertels@ucll.be>)
Ответы Re: viewing connectioninfo used by subscriber on the publicationserver when inactive  (Scott Ribe <scott_ribe@elevated-dev.com>)
Re: viewing connectioninfo used by subscriber on the publicationserver when inactive  (Wim Bertels <wim.bertels@ucll.be>)
Список pgsql-admin


On Thu, May 14, 2020 at 10:22 AM Wim Bertels <wim.bertels@ucll.be> wrote:
Keith Fiske schreef op do 14-05-2020 om 10:08 [-0400]:
> It doesn't matter how small the dataset change is. The same WAL
> stream is used for both logical and physical replication so it has to
> keep all WAL files until all subscribers for that publication have
> confirmed they have received them. If even a single subscriber goes
> offline, all WAL will be kept until that subscriber reconnects.

That is interesting, i assume this the WAL for the whole cluster, as
logical decoding is then used on this WAL for the logical replication,
do you have an estimate of order of magnitude for the all WAL files?

So far this seems ok over here (with one subscriber inactive for 2
days):
# du -ch pg_logical/ pg_wal/ pg_replslot/
912K    pg_logical/snapshots
4,0K    pg_logical/mappings
924K    pg_logical/
4,0K    pg_wal/archive_status
81M     pg_wal/
8,0K    pg_replslot/db2_sub
8,0K    pg_replslot/db2_sub1
8,0K    pg_replslot/db2_sub2
28K     pg_replslot/
81M     totaal
this after two days of replication setup.

assuming that students will be offline for at most 2 or 3 days,
this seems ok?

--
mvg,
Wim
--
Tell the truth or trump--but get the trick.
                -- Mark Twain, "Pudd'nhead Wilson's Calendar"



WAL is always cluster/instance-wide. Locally generated WAL files are, by default, always 16MB in size no matter how little has changed. So if you have archive_timeout set and only a single row has changed, that WAL file will still be 16MB (although it will compress down very small if you're archiving elsewhere).

How much of an impact this will be is entirely dependent on the write rate of your cluster. If you have very few writes you may be fine. But I would definitely suggest getting some monitoring in place if you expect to have offline subscribers for any long period of time.

But, again, I would still try and rethink this strategy. Offline subscribers can be a very big problem if they never come back. Not only because you'd eventually fill up your disk, but also because that no longer allows PG to recycle its WAL files, or can cause excessive cleanup operations when that subscriber finally comes back, which can have big IO impacts.
--
Keith Fiske
Senior Database Engineer
Crunchy Data - http://crunchydata.com

В списке pgsql-admin по дате отправления:

Предыдущее
От: Wim Bertels
Дата:
Сообщение: Re: viewing connectioninfo used by subscriber on the publicationserver when inactive
Следующее
От: Scott Ribe
Дата:
Сообщение: Re: viewing connectioninfo used by subscriber on the publicationserver when inactive