Re: Massive I/O spikes during checkpoint

От: David Kerr
Тема: Re: Massive I/O spikes during checkpoint
Дата: ,
Msg-id: 4FFBC90C.1060205@mr-paradox.net
(см: обсуждение, исходный текст)
Ответ на: Re: Massive I/O spikes during checkpoint  (Maxim Boguk)
Список: pgsql-performance

Скрыть дерево обсуждения

Massive I/O spikes during checkpoint  (David Kerr, )
 Re: Massive I/O spikes during checkpoint  (Maxim Boguk, )
  Re: Massive I/O spikes during checkpoint  (David Kerr, )
   Re: Massive I/O spikes during checkpoint  (Maxim Boguk, )
    Re: Massive I/O spikes during checkpoint  (David Kerr, )
    Re: Massive I/O spikes during checkpoint  (Andres Freund, )
     Re: Massive I/O spikes during checkpoint  (Jeff Janes, )
      Re: Massive I/O spikes during checkpoint  (Andres Freund, )
 Re: Massive I/O spikes during checkpoint  (Jeff Janes, )
  Re: Massive I/O spikes during checkpoint  (David Kerr, )

On 7/9/2012 11:14 PM, Maxim Boguk wrote:
>
>
> On Tue, Jul 10, 2012 at 4:03 PM, David Kerr <
> <mailto:>> wrote:
>
>
>     On Jul 9, 2012, at 10:51 PM, Maxim Boguk wrote:
>
>>
>>
>>         But what appears to be happening is that all of the data is
>>         being written out at the end of the checkpoint.
>>
>>         This happens at every checkpoint while the system is under load.
>>
>>         I get the feeling that this isn't the correct behavior and
>>         i've done something wrong.
>>
>>
>>
>>     It's not an actual checkpoints.
>>     It's is a fsync after checkpoint which create write spikes hurting
>>     server.
>>     You should set sysctl vm.dirty_background_bytes and vm.dirty_bytes
>>     to reasonable low values
>
>     So use bla_bytes instead of bla_ratio?
>
>
> Yes because on 256GB server
> echo 10 > /proc/sys/vm/dirty_ratio
> is equivalent to 26Gb dirty_bytes
>
> and
> echo 5 >/proc/sys/vm/dirty_background_ratio
> is equivalent to 13Gb dirty_background_bytes
>
> It is really huge values.
<sigh> yeah, I never bothered to think that through.

> So kernel doesn't start write any pages out in background before it has
> at least 13Gb dirty pages in kernel memory.
> And at end of the checkpoint kernel trying flush all dirty pages to disk.
>
> Even echo 1 >/proc/sys/vm/dirty_background_ratio  is too high value for
> contemporary server.
> That is why  *_bytes controls added to kernel.

Awesome, Thanks.


В списке pgsql-performance по дате сообщения:

От: Pena Kupen
Дата:
Сообщение: Fw: Re: Custom function in where clause
От: Andres Freund
Дата:
Сообщение: Re: Massive I/O spikes during checkpoint