RE: Recovery performance of standby for multiple concurrenttruncates on large tables

Поиск
Список
Период
Сортировка
От Jamison, Kirk
Тема RE: Recovery performance of standby for multiple concurrenttruncates on large tables
Дата
Msg-id D09B13F772D2274BB348A310EE3027C631D7D3@g01jpexmbkw24
обсуждение исходный текст
Ответ на Re: Recovery performance of standby for multiple concurrenttruncates on large tables  ('Andres Freund' <andres@anarazel.de>)
Ответы Re: Recovery performance of standby for multiple concurrent truncateson large tables  (Robert Haas <robertmhaas@gmail.com>)
Re: Recovery performance of standby for multiple concurrenttruncates on large tables  ('Andres Freund' <andres@anarazel.de>)
Список pgsql-hackers
Hi Andres,

> I think this is a case where the potential work arounds are complex enough to use significant resources to get right,
andare likely to make properly fixing the issue harder. I'm willing to comment on proposals that claim not to be
problmaticin those regards, but I have *SERIOUS* doubts they exist.
 

Alright. But I'd still try and ask your thoughts about it (below).
The proposed design touches the buffer invalidation during recovery process of standby server.

The question was about "how" to remember those buffers that contain truncate/drop tables, right?

1. Because the multiple scans of the whole shared buffer per concurrent truncate/drop table was the cause of the
time-consumingbehavior, DURING the failover process while WAL is being applied, we temporary delay the scanning and
invalidatingof shared buffers. At the same time, we remember the relations/relfilenodes (of dropped/truncated tables)
byadding them in a hash table called "skip list". 
 
2. After WAL is applied, the checkpoint(or bg writer) scans the shared buffer only ONCE, compare the pages against the
skiplist, and invalidates the relevant pages. After deleting the relevant pages on the shared memory, it will not be
writtenback to the disk.
 

Assuming the theory works, this design will only affect the behavior of checkpointer (or maybe bg writer) during
recoveryprocess / failover. Any feedback, thoughts?
 

BTW, are there any updates whether the community will push through anytime soon regarding the buffer mapping
implementationyou mentioned?
 


Regards,
Kirk Jamison



В списке pgsql-hackers по дате отправления:

Предыдущее
От: Melanie Plageman
Дата:
Сообщение: Re: Making "COPY partitioned_table FROM" faster
Следующее
От: Kyotaro HORIGUCHI
Дата:
Сообщение: Re: Usability fail with psql's \dp command