Re: BUG #12071: Stat collector went crasy (50MB/s constant writes)

On 27.11.2014 00:01, Maxim Boguk wrote:
>
>
> On Thu, Nov 27, 2014 at 1:26 AM, Tomas Vondra <tv@fuzzy.cz
> <mailto:tv@fuzzy.cz>> wrote:
>
>     On 26.11.2014 23:06, Maxim Boguk wrote:
>     >
>     >
>     >     I'm not sure I understand what you mean.
>     >
>     >     The timestamp *IS* updated on every write - see this:
>     >
>     >     https://github.com/postgres/postgres/blob/REL9_2_STABLE/src/backend/postmaster/pgstat.c#L3469
>     >
>     >
>     >  I mean that usually 99.9% writes goes through mmap, which update the
>     > pgstat.tmp file timestamp (but not update last_statwrite value).
>
>     I don't follow. Can you explain what you mean by the mmap? I don't see
>     any mmap stuff in pgstat.c, nor I remember dealing with mmap in my
>     patches in this area ...
>
>
> Hm I had been wrong. I seen mmap call on strace after every statistic
> collector file rewrite and I thought that all writes to that file goes
> through memory mapped structure instead of full file writes every time
> when someone need fresh data.
>
> However, point still open - there are no reason to write a new file
> from scratch if no activity had been received from stat socket since
> the last write (the same point valid for per-database stat files in
> 9.3+).

Well, nothing is perfect and patches are always welcome ;-)

For us, the per-db split was enough - it lowered the CPU and I/O load to
the point that adding such 'is dirty' flag would make absolutely no
difference. I do have it on my TODO list, but it's not very high.

> And check
> https://github.com/postgres/postgres/blob/REL9_2_STABLE/src/backend/postmaster/pgstat.c#L3972
> looks like redundant in that case?

Redundant with what?

That's how the backend checks whether the new file has been written (by
postmaster, i.e. a different process). Keep in mind that this happens in
different process than

    if (last_statwrite < last_statrequest)
        pgstat_write_statsfile(false);

which runs in the postmaster (and not in the backend).

Tomas