Re: fix log_min_duration_statement logic error

Tom Lane wrote:
> Peter Eisentraut <peter_e@gmx.net> writes:
> > When you're dealing with a large installation, these little Perl scripts
> > become difficult.  I've just had to deal with a similar issue with a
> > popular MTA which spreads the relevant log information over several lines.
> > If you're generating 500 MB of log output a day, it becomes a problem.
>
> True, but it would take really serious revisions of our log output
> formatting to fix things so that a naive "grep" will be useful for much
> of anything.
>
> To take just the most obvious limitation in what Bruce is proposing: a
> grep for "duration" will yield the duration and the first line of the
> SQL command.  If an installation is in the habit of breaking their SQL
> into multiple lines, this will be less than useful.  Some people like
> to format their code like this:
>
>     SELECT
>         a, b, c
>     FROM
>         ...
>
> in which case showing just the first line will be quite content-free.
>

It is pretty easy to continue pulling lines after the 'duration' hit to
see the full query, perhap using awk, or even grep with a + arg.  I just
don't see why we should make it harder for folks by splitting it over
several lines.  For folks that want to use perl, they can use the
existing log_* outputs to join by pid and get whatever they want.  If
you want to call this option an option for dummies that don't want to
deal with Perl (and Peter has expressed that it isn't easy), that's
fine.

Does someone want to show us a Perl script do join lines for this
must-requested output?  I don't feel like sending it to everyone who
needs to find the slow queries in their application.  Do you?

If you want, we can have vote on it.  Frankly, the code was designed and
submitted as one line output, and I think a vote will show one-line
output as the winner.

> There are already similar problems with extracting error information
> from the logs (and the 7.4 redesign of error formatting has made 'em
> worse).
>
> If we are going to try to design the log output so that you don't need
> reassembly scripts to link related lines together, then we have got lots
> bigger problems to fix than log_duration.  I'm not convinced it's an
> appropriate goal at all, though.

Again, it is cost/benefit --- how many people want easy reporting of
queries and their durations.

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073