Re: checkpointer continuous flushing

On Sat, Jan 9, 2016 at 7:10 PM, Andres Freund <andres@anarazel.de> wrote:
>
> On 2016-01-09 19:05:54 +0530, Amit Kapila wrote:
> > Right that won't be acceptable, however I think with your latest
> > proposal [1]
>
> Sure, that'd address that problem.
>
>
> > [...] think that idea will help to mitigate the problem of backend and
> > bgwriter writes as well.  In that, can't we do it with the help of
> > existing infrastructure of *pendingOpsTable* and
> > *CheckpointerShmem->requests[]*, as already the flush requests are
> > remembered in those structures, we can use those to apply your idea to
> > issue flush requests.
>
> Hm, that might be possible. But that might have some bigger implications
> - we currently can issue thousands of flush requests a second, without
> much chance of merging. I'm not sure it's a good idea to overlay that
> into the lower frequency pendingOpsTable.

>

In that case, we can have unified structure to remember flush requests

rather than backend and bgwriter noting that information in

CheckpointerShmem and checkpointer in pendingOpsTable.  I understand

there are some benefits of having pendingOpsTable, but having a

common structure seems to be more beneficial and in particular

because it can be used for the purpose of flush hints.

Now, I am sure we can invent a new way of tracking the flush

requests for flush hints, but I think we might want to consider why

can't we have one unified way of tracking the flush requests which

can be used both for *flush* and *flush hints*.

> Backends having to issue
> fsyncs because the pending fsync queue is full is darn expensive. In
> contrast to that a 'flush hint' request getting lost doesn't cost that
> much.
>