On Wed, 2008-12-24 at 15:51 +0530, Pavan Deolasee wrote:
> On Wed, Dec 24, 2008 at 3:40 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
> >
> >
> > If we want to speed up recovery more, I think we'll see the need for an
> > additional process to do WAL CRC checks.
> >
>
> Yeah, any such helper process along with other optimizations would
> certainly help. But I can't still believe that on a high load, high
> end setup, single recovery process without any read-ahead for data
> blocks, can keep pace with the WAL generated by hundreds of processes
> at the primary and shipped over a high speed link to standby.
Suzuki-san has provided measurements. I think we need more. With
bgwriter performing restartpoints, we'll find that more RAM helps much
more than it did previously.
> BTW, on a completely different note, given that the entire recovery is
> based on physical redo, are there any inherent limitations that we
> can't do parallel recovery where different recovery processes apply
> redo logs to completely independent set of data blocks ?
That's possible, but will significantly complicate the recovery code.
Retaining the ability to do standby queries would be almost impossible
in that case, since you would need to parallelise the WAL stream without
changing the commit order of transactions.
The main CPU bottleneck is CRC, by a long way. If we move effort away
from startup process that is the best next action, AFAICS.
> I also
> sometimes wonder why we don't have block level recovery when a single
> block in the database is corrupted. Can't this be done by just
> selectively applying WAL records to that particular block ? If it's
> just because nobody had time/interest to do this, then it's OK, but I
> wonder if there are any design issues.
You'll be able to do this with my rmgr patch. Selective application of
WAL records is one of the primary use cases, but there are many others.
-- Simon Riggs www.2ndQuadrant.comPostgreSQL Training, Services and Support