Re: Hot standby, recovery infra

Simon Riggs wrote:
> On Thu, 2009-02-05 at 10:31 +0200, Heikki Linnakangas wrote:
>> Simon Riggs wrote:
>>> On Thu, 2009-02-05 at 09:28 +0200, Heikki Linnakangas wrote:
>>>> I got rid of minSafeStartPoint, advancing minRecoveryPoint instead. And 
>>>> it's advanced in XLogFlush instead of XLogFileRead. I'll post an updated 
>>>> patch soon.
>>> Why do you think XLogFlush is called less frequently than XLogFileRead?
>> It's not, but we only need to update the control file when we're 
>> "flushing" an LSN that's greater than current minRecoveryPoint. And when 
>> we do update minRecoveryPoint, we can update it to the LSN of the last 
>> record we've read from the archive.
> 
> So we might end up flushing more often *and* we will be doing it
> potentially in the code path of other users.

For example, imagine a database that fits completely in shared buffers. 
If we update at every XLogFileRead, we have to fsync every 16MB of WAL. 
If we update in XLogFlush the way I described, you only need to update 
when we flush a page from the buffer cache, which will only happen at 
restartpoints. That's far less updates.

Expanding that example to a database that doesn't fit in cache, you're 
still replacing pages from the buffer cache that have been untouched for 
longest. Such pages will have an old LSN, too, so we shouldn't need to 
update very often.

I'm sure you can come up with an example of where we end up fsyncing 
more often, but it doesn't seem like the common case to me.

> This change seems speculative and also against what has previously been
> agreed with Tom. If he chooses not to comment on your changes, that's up
> to him, but I don't think you should remove things quietly that have
> been put there through the community process, as if they caused
> problems. I feel like I'm in the middle here. 

I'd like to have the extra protection that this approach gives. If we 
let safeStartPoint to be ahead of the actual WAL we've replayed, we have 
to just assume we're fine if we reach end of WAL before reaching that 
point. That assumption falls down if e.g recovery is stopped, and you go 
and remove the last few WAL segments from the archive before restarting 
it, or signal pg_standby to trigger failover too early. Tracking the 
real safe starting point and enforcing it always protects you from that.

(we did discuss this a week ago: 
http://archives.postgresql.org/message-id/4981F6E0.2040503@enterprisedb.com)

--   Heikki Linnakangas  EnterpriseDB   http://www.enterprisedb.com