Обсуждение: Re: [PERFORM] WAL Optimisation - configuration and usage

>Neil Conway
> Simon Riggs wrote:
> >>Josh Berkus wrote
> >>
> >>>Simon Riggs wrote
> >>>Please set WAL_DEBUG to 1 so we can see a bit more info: thanks.
> >>
> >>I'm pretty sure that WAL_DEBUG requires a compile-time option.
> >
> > I'm surprised, but you are right, the manual does SAY this requires
a
> > compile time option; it is unfortunately not correct.
>
> Actually, the manual is correct: in 7.4 and earlier releases, enabling
> wal_debug can be done without also setting a compile-time #ifdef. As
> of current CVS HEAD, the WAL_DEBUG #ifdef must be defined before this
> variable is available.

Touche! I stand corrected, thank you both. My suggestion does work for
Rob, then.

[This also implies I have a screwed version on my machine, so thank you
also for flushing that lurking issue out for me. I'd had a suspicion for
a few weeks. Lucky I'm still just prototyping.]

On the other hand, I was just about to change the wal_debug behaviour to
allow better debugging of PITR features as they're added. I think it is
very important to be able to put the system fairly easily into debug
mode; a recompile is easy enough, but it would be even better to avoid
this completely. This would mean reversing the change you describe:
here's the design:

The behaviour I wish to add is:
Keep wal_debug as a value between 0 and 16.
If =0 then no debug output (default).
Use following bitmasks against the value
Mask 1 = XLOG Checkpoints get logged
Mask 2 = Archive API calls get logged
Mask 4 = Transaction - commits get logged
Mask 8 = Flush & INSERTs get logged

That way it should be fairly straightforward to control the amount and
type of information available to administrators. The existing design
produces too much info to be easily usable, mostly requiring a perl
program to filter out the info overload and do record counts. This
suggested design allows you to control the volume of messages, since the
bitmasks are arranged in volume/frequency order and brings the wal_debug
option back into something useful for problem diagnosis on live systems,
not just hacking the code.

Anybody object to these mods, or have better/different ideas? Getting
the diagnostics right is fairly important, IMHO, to making PITR become
real.

Best regards, Simon Riggs

"Simon Riggs" <simon@2ndquadrant.com> writes:
> The behaviour I wish to add is:
> Keep wal_debug as a value between 0 and 16.
> If =0 then no debug output (default).
> Use following bitmasks against the value
> Mask 1 = XLOG Checkpoints get logged
> Mask 2 = Archive API calls get logged
> Mask 4 = Transaction - commits get logged
> Mask 8 = Flush & INSERTs get logged

I see no value in reverting Neil's change.  The above looks way too much
like old-line assembler-programmer thinking to me, anyway.  Why not
invent a separate, appropriately named boolean variable for each thing
you want to control?  Even C programmers manage to avoid doing the sort
of mental arithmetic that the above would force onto DBAs.

As for whether it should be #ifdef'd or not, I'd have no objection to
turning WAL_DEBUG on by default in pg_config_manual.h for the duration
of PITR development.  One should not however confuse short-term
debugging needs with features that the average user is going to need
indefinitely.  (It was not too long ago that there was still debugging
code for btree index building in there, for crissakes.)

            regards, tom lane

Simon Riggs wrote:
> On the other hand, I was just about to change the wal_debug behaviour to
> allow better debugging of PITR features as they're added.

That's a development activity. Enabling the WAL_DEBUG #ifdef by
default during the 7.5 development cycle would be uncontroversial, I
think.

> I think it is very important to be able to put the system fairly
> easily into debug mode

It is? Why would this be useful for non-development activities?

(It may well be the case that we ought to report more or better
information about the status of the WAL subsystem; but WAL_DEBUG is
surely not the right mechanism for emitting information intended for
an administrator.)

-Neil

>Tom Lane
> "Simon Riggs" <simon@2ndquadrant.com> writes:
> > The behaviour I wish to add is:
> > Keep wal_debug as a value between 0 and 16.
> > If =0 then no debug output (default).
> > Use following bitmasks against the value
> > Mask 1 = XLOG Checkpoints get logged
> > Mask 2 = Archive API calls get logged
> > Mask 4 = Transaction - commits get logged
> > Mask 8 = Flush & INSERTs get logged
> 
> I see no value in reverting Neil's change.  The above looks way too
much
> like old-line assembler-programmer thinking to me, anyway.  Why not
> invent a separate, appropriately named boolean variable for each thing
> you want to control?  Even C programmers manage to avoid doing the
sort
> of mental arithmetic that the above would force onto DBAs.
> 
> As for whether it should be #ifdef'd or not, I'd have no objection to
> turning WAL_DEBUG on by default in pg_config_manual.h for the duration
> of PITR development.  One should not however confuse short-term
> debugging needs with features that the average user is going to need
> indefinitely.  (It was not too long ago that there was still debugging
> code for btree index building in there, for crissakes.)

...erm, I guess you didn't like that one then? ;}

> As for whether it should be #ifdef'd or not, I'd have no objection to
> turning WAL_DEBUG on by default in pg_config_manual.h for the duration
> of PITR development.  

Yes OK, thank you.

> Why not
> invent a separate, appropriately named boolean variable for each thing
> you want to control?

Yes, OK, will do.

Best Regards, Simon Riggs

>Neil Conway
> Simon Riggs wrote:
> > On the other hand, I was just about to change the wal_debug
behaviour to
> > allow better debugging of PITR features as they're added.
>
> That's a development activity. Enabling the WAL_DEBUG #ifdef by
> default during the 7.5 development cycle would be uncontroversial, I
> think.

Yes that's the best proposal. Can I leave that with you?

> > I think it is very important to be able to put the system fairly
> > easily into debug mode
>
> It is? Why would this be useful for non-development activities?
>
> (It may well be the case that we ought to report more or better
> information about the status of the WAL subsystem; but WAL_DEBUG is
> surely not the right mechanism for emitting information intended for
> an administrator.)

Right again. I guess my proposal amounted to quick-and-dirty logging.

I'll think some more.

Best Regards, Simon Riggs