Re: Potential Large Performance Gain in WAL synching

Bruce Momjian wrote:
> I may be missing something here, but other backends don't block while
> one writes to WAL.

I don't think they'll block until they get to the fsync or XLogWrite
call while another transaction is fsync'ing.

I'm no Unix filesystem expert but I don't see how the OS can
handle multiple writes and fsyncs to the same file descriptors without
blocking other processes from writing at the same time. It may be that
there are some clever data structures they use but I've not seen huge
praise for most of the file systems. A well written file system could
minimize this contention but I'll bet it's there with most of the ones
that PostgreSQL most commonly runs on.

I'll have to write a test and see if there really is a problem.

- Curtis

> -----Original Message-----
> From: Bruce Momjian [mailto:pgman@candle.pha.pa.us]
> Sent: Friday, October 04, 2002 12:44 AM
> To: Curtis Faith
> Cc: Tom Lane; Pgsql-Hackers
> Subject: Re: [HACKERS] Potential Large Performance Gain in WAL synching
>
>
> Curtis Faith wrote:
> > The method I propose does not result in any blocking because of writes
> > other than the final commit's write and it has the very significant
> > advantage of allowing other transactions (from other back-ends) to
> > continue until they enter commit (and blocking waiting for their final
> > commit write to complete).
> >
> > > > 2) Allow transactions to complete and do work while other
> threads are
> > > > waiting on the completion of the log write.
> > >
> > > I'm missing something.  There is no useful work that a transaction can
> > > do between writing its commit record and reporting
> completion, is there?
> > > It has to wait for that record to hit disk.
> >
> > The key here is that a thread that has not committed and therefore is
> > not blocking can do work while "other threads" (should have
> said back-ends
> > or processes) are waiting on their commit writes.
>
> I may be missing something here, but other backends don't block while
> one writes to WAL.  Remember, we are proccess based, not thread based,
> so the write() call only blocks the one session.  If you had threads,
> and you did a write() call that blocked other threads, I can see where
> your idea would be good, and where async i/o becomes an advantage.
>
> --
>   Bruce Momjian                        |  http://candle.pha.pa.us
>   pgman@candle.pha.pa.us               |  (610) 359-1001
>   +  If your life is a hard drive,     |  13 Roberts Road
>   +  Christ can be your backup.        |  Newtown Square,
> Pennsylvania 19073
>