Re: Improvement of checkpoint IO scheduler for stable transaction responses

Поиск
Список
Период
Сортировка
От Pavan Deolasee
Тема Re: Improvement of checkpoint IO scheduler for stable transaction responses
Дата
Msg-id CABOikdMrQqPy9Mr=UWeg=8K9G0SkWmW0tD4WTXdup5xfemjvMQ@mail.gmail.com
обсуждение исходный текст
Ответ на Re: Improvement of checkpoint IO scheduler for stable transaction responses  (Andres Freund <andres@2ndquadrant.com>)
Список pgsql-hackers



On Mon, Jun 17, 2013 at 2:18 AM, Andres Freund <andres@2ndquadrant.com> wrote:
On 2013-06-16 17:27:56 +0300, Heikki Linnakangas wrote:

> A long time ago, Itagaki wrote a patch to sort the checkpoint writes: www.postgresql.org/message-id/flat/20070614153758.6A62.ITAGAKI.TAKAHIRO@oss.ntt.co.jp.
> He posted very promising performance numbers, but it was dropped because Tom
> couldn't reproduce the numbers, and because sorting requires allocating a
> large array, which has the risk of running out of memory, which would be bad
> when you're trying to checkpoint.

Hm. We could allocate the array early on since the number of buffers
doesn't change. Sure that would be pessimistic, but that seems fine.

Alternatively I can very well imagine that it would still be beneficial
to sort the dirty buffers in shared buffers. I.e. scan till we found 50k
dirty pages, sort them and only then write them out.


Without knowing that Itagaki had done something similar in the past, couple of months back I tried exactly the same thing i.e. sort the shared buffers in chunks and then write them out at once. But I did not get any significant performance gain except when the shared buffers are 3/4th (or some such number) or more than the available RAM. I will see if I can pull out the patch and the numbers. But if memory serves well, I concluded that the kernel is already utilising its buffer cache to achieve the same thing and it does not help beyond a point.

Thanks,
Pavan

--
Pavan Deolasee
http://www.linkedin.com/in/pavandeolasee

В списке pgsql-hackers по дате отправления:

Предыдущее
От: Simon Riggs
Дата:
Сообщение: Re: refresh materialized view concurrently
Следующее
От: KONDO Mitsumasa
Дата:
Сообщение: Re: Improvement of checkpoint IO scheduler for stable transaction responses