RE: Recovery performance of standby for multiple concurrenttruncates on large tables

Поиск
Список
Период
Сортировка
От Jamison, Kirk
Тема RE: Recovery performance of standby for multiple concurrenttruncates on large tables
Дата
Msg-id D09B13F772D2274BB348A310EE3027C6300333@g01jpexmbkw24
обсуждение исходный текст
Ответ на Re: Recovery performance of standby for multiple concurrenttruncates on large tables  (Andres Freund <andres@anarazel.de>)
Ответы Re: Recovery performance of standby for multiple concurrenttruncates on large tables  ('Andres Freund' <andres@anarazel.de>)
Список pgsql-hackers

Hi,

Thank you for your replies.

 

On Tue, July 10, 2018 4:15 PM, Andres Freund wrote:

>I think you'd run into a lot of very hairy details with this approach. Consider what happens if client processes need fresh buffers and need to write out a victim buffer. You'll need to know that the relevant buffer is actually invalid. Thus the knowledge about the "delayed" drops would need to be in shared buffers and scanned on every dirty buffer writeout.

 

Yes, you're right. There are problems associated with checkpoint as you pointed out. I just thought of touching the involved process on writing dirty pages to the disk, which are both the functions of checkpoint and background writer.

> How about using the background writer for this? ...

> ...

And now that I think about it, the suggestion of Ants above on "background writer" would seem to work better, as bg writer is more active and continuously write dirty pages but checkpointer only does it in interval. This seems to be a more viable solution and I'd appreciate advice. I'll explore around this idea, and if possible, I'd like to develop a solution for the next commitfest.

 

> I personally think we should rather just work towards a ordered buffer mapping implementation.

 

I understand that the main problem lies on shared_buffers scanning and that a buffer mapping implementation is underway(?) for PG12. I am not sure if the community has arrived at a consensus for that long-term fix. Would the community also welcome any minor/smaller-scale improvements (just for this particular problem on wal recovery for truncate table)? If yes, then I'd like to work on a possible solution.

 

Regards,

Kirk Jamison

В списке pgsql-hackers по дате отправления:

Предыдущее
От: Asim R P
Дата:
Сообщение: Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket
Следующее
От: Michael Paquier
Дата:
Сообщение: Re: print_path is missing GatherMerge and CustomScan support