Re: max_standby_delay considered harmful

On Tue, 2010-05-04 at 09:12 -0400, Stephen Frost wrote:
> * Simon Riggs (simon@2ndQuadrant.com) wrote:
> > If recovery waits for max_standby_delay every time something gets in its
> > way, it should be clear that if many things get in its way it will
> > progressively fall behind. There is no limit to this and it can always
> > fall further behind. It does result in fewer cancelled queries and I do
> > understand many may like that.
> 
> Guess I wasn't very clear in my previous description of what I *think*
> the change would be (Tom, please jump in if I've got this wrong..).
> Recovery wouldn't wait max_standby_delay every time; I agree, that would
> be a big change in behaviour and could make it very difficult for the
> slave to keep up.  Rather, recovery would proceed as normal until it
> encounters a lock, at which point it would start a counting down from
> max_standby_delay, if the lock is released before it hits that, then it
> will move on, if another lock is encoutered, it would start counting
> down from where it left off last time.  If it hits zero, it'll cancel
> the other query, and any other queries that get in the way, until it's
> caught up again completely.  Once recovery is fully caught up, the
> counter would reset again to max_standby_delay.

This new clarification is almost exactly how it works already. Sounds
like the existing docs need some improvement.

The only difference is that max_standby_delay is measured from log
timestamp. Perhaps it should work from WAL receipt timestamp rather than
from log timestamp? That would make some of the problems go away without
significantly changing the definition. I'll look at that.

(And that conflicts are caused by more situations than just locks, but
that detail doesn't alter your point).

> > The bottom line is this is about conflict resolution. There is simply no
> > way to resolve conflicts without favouring one or other of the
> > protagonists. Whatever mechanism you come up with that favours one will,
> > disfavour the other. I'm happy to give choices, but I'm not happy to
> > force just one kind of conflict resolution.
> 
> I don't think anyone is trying to get rid of the knob entirely; you're
> right, you can't please everyone all the time, so there has to be some
> kind of knob there which people can adjust based on their particular use
> case and system.  This is about what exactly the knob is and how it's
> implemented and documented.

I'm happy with more than one way. It'd be nice if a single parameter,
giving one dimension of tuning, suited all ways people have said they
would like it to behave. I've not found a way of doing that.

I have no problem at all with adding additional parameters or mechanisms
to cater for the multiple dimensions of control people have asked for.
So your original interpretation is also valid for some users.

-- Simon Riggs           www.2ndQuadrant.com