Re: Reducing stats collection overhead

Поиск
Список
Период
Сортировка
От Bruce Momjian
Тема Re: Reducing stats collection overhead
Дата
Msg-id 200704290717.l3T7HZB08040@momjian.us
обсуждение исходный текст
Ответ на Reducing stats collection overhead  (Tom Lane <tgl@sss.pgh.pa.us>)
Ответы Re: Reducing stats collection overhead  (Tom Lane <tgl@sss.pgh.pa.us>)
Список pgsql-hackers
Yes, it seems we will have to do something for 8.3.  I assume the method
below would reduce frequent updates of the stats_command_string too.

---------------------------------------------------------------------------

Tom Lane wrote:
> Arjen van der Meijden told me that according to the tweakers.net
> benchmark, HEAD is noticeably slower than 8.2.4, and I soon confirmed
> here that for small SELECT queries issued as separate transactions,
> there's a significant difference.  I think much of the difference stems
> from the fact that we now have stats_row_level ON by default, and so
> every transaction sends a stats message that wasn't there by default
> in 8.2.  When you're doing a few thousand transactions per second
> (not hard for small read-only queries) that adds up.
> 
> It seems to me that this could be fixed fairly easily by allowing the
> stats to accumulate across multiple small transactions before sending
> a message.  There's surely not much point in kicking stats out quickly
> when the stats collector only reports them to the world every half
> second anyway.
> 
> The first design that comes to mind is that at transaction end
> (pgstat_report_tabstat() time) we send a stats message only if at least
> X milliseconds have elapsed since we last sent one, where X is
> PGSTAT_STAT_INTERVAL or closely related to it.  We also make sure to
> flush stats out before process exit.  This approach ensures that in a
> lots-of-short-transactions scenario, we only need to send one stats
> message every X msec, not one per query.  The cost is possible delay of
> stats reports.  I claim that any transaction that makes a really sizable
> change in the stats will run longer than X msec and therefore will send
> its stats immediately.  Cases where a client does a small transaction
> after sleeping for awhile (more than X msec) will also send immediately.
> You might get a delay in reporting the last few transactions of a burst
> of short transactions, but how much does it matter?  So I think that
> complicating the design with, say, a timeout counter to force out the
> stats after a sleep interval is not necessary.  Doing so would add a
> couple of kernel calls to every client interaction so I'd really rather
> avoid that.
> 
> Any thoughts, better ideas?
> 
>             regards, tom lane
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 3: Have you checked our extensive FAQ?
> 
>                http://www.postgresql.org/docs/faq

--  Bruce Momjian  <bruce@momjian.us>          http://momjian.us EnterpriseDB
http://www.enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +


В списке pgsql-hackers по дате отправления:

Предыдущее
От: "Pavel Stehule"
Дата:
Сообщение: Re: pgsql crollable cursor doesn't support one formofpostgresql's cu
Следующее
От: Tom Lane
Дата:
Сообщение: Re: Reducing stats collection overhead