Обсуждение: Should pg_stat_bgwriter.buffers_backend_fsync be removed?
Backend fsyncs are theoretically still possible after the fsync request queue compaction patch (which was subsequently back-patched to all supported release branches). However, I'm reasonably confident that that patch was so effective as to make a backend fsync all but impossible. As such, it seems like the buffers_backend_fsync column in the pg_stat_bgwriter view is more or less obsolete. I suggest removing it for 9.5, and instead logging individual occurrences of backend fsync requests within ForwardFsyncRequest(). It seems fair to treat that as an anomaly to draw particular attention to. -- Peter Geoghegan
Peter Geoghegan <pg@heroku.com> writes:
> Backend fsyncs are theoretically still possible after the fsync
> request queue compaction patch (which was subsequently back-patched to
> all supported release branches). However, I'm reasonably confident
> that that patch was so effective as to make a backend fsync all but
> impossible.
What's your evidence for that claim?
regards, tom lane
On Sat, Apr 26, 2014 at 9:16 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Peter Geoghegan <pg@heroku.com> writes: >> Backend fsyncs are theoretically still possible after the fsync >> request queue compaction patch (which was subsequently back-patched to >> all supported release branches). However, I'm reasonably confident >> that that patch was so effective as to make a backend fsync all but >> impossible. > > What's your evidence for that claim? I don't have any evidence, but I think it unlikely that this is an occurrence that is seen in the real world. I was not able to see any instances of it on the entire Heroku fleet at one point a few months back, for one thing. For another, I have never observed this with any benchmark, even though pgbench-tools presents buffers_backend_fsync for each test run. The queue compaction patch completely fixed Greg Smith's original test case. Even then, I believe it was considered more of a patch addressing an edge case than anything else. Even the comments above ForwardFsyncRequest() consider the occurance of backend fsyncs to be only "theoretically possible". If anyone is aware of any cases where this is still actually known to happen in production, I'd like to hear about them. However, ISTM that if this actually does still happen, those cases would be better served by surfacing the problem in the logs. -- Peter Geoghegan
--On 26. April 2014 19:42:47 -0700 Peter Geoghegan <pg@heroku.com> wrote: > I suggest removing it for 9.5, and instead logging individual > occurrences of backend fsync requests within ForwardFsyncRequest(). It > seems fair to treat that as an anomaly to draw particular attention > to. But wouldn't that make it more complicated/unlikely to discover cases, where it still doesn't work? -- Thanks Bernd
On Tue, Apr 29, 2014 at 2:51 AM, Bernd Helmle <mailings@oopsware.de> wrote: >> I suggest removing it for 9.5, and instead logging individual >> occurrences of backend fsync requests within ForwardFsyncRequest(). It >> seems fair to treat that as an anomaly to draw particular attention >> to. > > But wouldn't that make it more complicated/unlikely to discover cases, where > it still doesn't work? I don't think so, no. -- Peter Geoghegan
On Tue, Apr 29, 2014 at 5:54 AM, Peter Geoghegan <pg@heroku.com> wrote: > On Tue, Apr 29, 2014 at 2:51 AM, Bernd Helmle <mailings@oopsware.de> wrote: >>> I suggest removing it for 9.5, and instead logging individual >>> occurrences of backend fsync requests within ForwardFsyncRequest(). It >>> seems fair to treat that as an anomaly to draw particular attention >>> to. >> >> But wouldn't that make it more complicated/unlikely to discover cases, where >> it still doesn't work? > > I don't think so, no. I think it just depends. For people who are running a log scraper anyway, a message would be better than a statistics counter, because it's one less thing to check. For people who are running something that monitors the stats views anyway, but perhaps not a log scraper, the counter is better. Overall, I don't see much reason to tinker with this. If we had no reporting at all of this condition now, I'd probably be mildly more supportive of adding a log message than a counter. But since we've already got something and there's no real problem with it, I'm disinclined to make a change. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
Robert Haas <robertmhaas@gmail.com> writes:
> Overall, I don't see much reason to tinker with this. If we had no
> reporting at all of this condition now, I'd probably be mildly more
> supportive of adding a log message than a counter. But since we've
> already got something and there's no real problem with it, I'm
> disinclined to make a change.
+1 ... if it ain't broke, why fix it?
regards, tom lane
On 4/26/14, 9:42 PM, Peter Geoghegan wrote: > Backend fsyncs are theoretically still possible after the fsync > request queue compaction patch (which was subsequently back-patched to > all supported release branches). However, I'm reasonably confident > that that patch was so effective as to make a backend fsync all but > impossible. As such, it seems like the buffers_backend_fsync column in > the pg_stat_bgwriter view is more or less obsolete. > > I suggest removing it for 9.5, and instead logging individual > occurrences of backend fsync requests within ForwardFsyncRequest(). It > seems fair to treat that as an anomaly to draw particular attention > to. All else equal, I don't like the idea of removing this from pg_stat_bgwriter. Being able to look there and see if this isoccurring since last stats reset is much easier than grepping logfiles. I don't have an issue with logging it, though I think we need to be careful to ensure we don't go crazy if something happensin the system where suddenly all backends are fsyncing. If that happens you're going to have major IO problems andtrying to log thousands (or more) of extra entries is just going to make it worse. -- Jim C. Nasby, Data Architect jim@nasby.net 512.569.9461 (cell) http://jim.nasby.net
On Tue, Apr 29, 2014 at 12:02 PM, Jim Nasby <jim@nasby.net> wrote: > All else equal, I don't like the idea of removing this from > pg_stat_bgwriter. Being able to look there and see if this is occurring > since last stats reset is much easier than grepping logfiles. Have you ever actually seen it at a non-zero value after the compaction patch went in? -- Peter Geoghegan