Re: [HACKERS] log_destination=file

Поиск
Список
Период
Сортировка
От Magnus Hagander
Тема Re: [HACKERS] log_destination=file
Дата
Msg-id CABUevEygVLWbF5LmQWkGiPmeo7456rjyED0X3Uj1PtsY5XhEOQ@mail.gmail.com
обсуждение исходный текст
Ответ на Re: [HACKERS] log_destination=file  (Robert Haas <robertmhaas@gmail.com>)
Ответы Re: [HACKERS] log_destination=file
Re: [HACKERS] log_destination=file
Список pgsql-hackers


On Tue, Nov 14, 2017 at 5:33 PM, Robert Haas <robertmhaas@gmail.com> wrote:
On Sun, Sep 10, 2017 at 5:29 AM, Dmitry Dolgov <9erthalion6@gmail.com> wrote:
> average latency:
>
> clients     patch       master
> 10          0.321       0.286
> 20          0.669       0.602
> 30          1.016       0.942
> 40          1.358       1.280
> 50          1.727       1.637

That's still a noticeable slowdown, though.  And we've had previous
reports of the overhead of logging being significant as well:

http://postgr.es/m/CACLsApsA7U0GCFpojVQem6SGTEkv8vnwdBfhVi+dqO+gu5gdCA@mail.gmail.com

I seem to recall a discussion, perhaps in-person, around the time Theo
submitted that patch where it was reported that the logging collector
could not be used on some systems he was working with because it
became a major performance bottleneck.  With each backend writing its
own messages to a file, it was tolerable, but when you tried to funnel
everything through a single process, the back-pressure slowed down the
entire system unacceptably.

Finally found myself back at this one, because I still think this is a problem we definitely need to adress (whether with this file or not).

The funneling into a single process is definitely an issue.

But we don't really solve that problem today wit logging to stderr, do we? Because somebody has to pick up the log as it came from stderr. Yes, you get more overhead when sending the log to devnull, but that isn't really a realistic scenario. The question is what to do when you actually want to collect that much logging that quickly.

If each backend could actually log to *its own file*, then things would get sped up. But we can't do that today. Unless you use the hooks and build it yourself.

Per the thread referenced, using the hooks to handle the very-high-rate-logging case seems to be the conclusion. But is that still the conclusion, or do we feel we need to also have a native solution?

And if the conclusion is that hooks is the way to go for that, then is the slowdown of this patch actually a relevant problem to it?

--

В списке pgsql-hackers по дате отправления:

Предыдущее
От: Magnus Hagander
Дата:
Сообщение: Re: [HACKERS] Supporting huge pages on Windows
Следующее
От: Peter Eisentraut
Дата:
Сообщение: Re: improve type conversion of SPI_processed in Python