Обсуждение: appendPQExpBufferVA vs appendStringInfoVA

David Rowley <dgrowleyml@gmail.com> writes:
> Tom commited some changes to appendStringInfoVA a few weeks ago which
> allows it to return the required buffer size if the current buffer is not
> big enough.

> On looking at appendPQExpBufferVA I'm thinking it would be nice if it could
> make use of the new pvsnprintf function to bring the same potential
> performance improvement in to there too.

Uh ... it does contain pretty much the same algorithm now.  We can't
simply use pvsnprintf there because exit-on-error is no good for
libpq's purposes, so unless we want to rethink that, a certain
amount of code duplication is unavoidable.  But they both understand
about C99 vsnprintf semantics now.
        regards, tom lane

On Fri, Nov 1, 2013 at 9:27 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> David Rowley <dgrowleyml@gmail.com> writes:
>> Tom commited some changes to appendStringInfoVA a few weeks ago which
>> allows it to return the required buffer size if the current buffer is not
>> big enough.
>
>> On looking at appendPQExpBufferVA I'm thinking it would be nice if it could
>> make use of the new pvsnprintf function to bring the same potential
>> performance improvement in to there too.
>
> Uh ... it does contain pretty much the same algorithm now.  We can't
> simply use pvsnprintf there because exit-on-error is no good for
> libpq's purposes, so unless we want to rethink that, a certain
> amount of code duplication is unavoidable.  But they both understand
> about C99 vsnprintf semantics now.

I have often found it frustrating that we have appendStringInfo* for
the backend and appendPQExpBuffer* for the frontend.  It'd be nice to
have one API that could be used in both places, somehow.  There seems
to be a lot of interest (including on my part) in writing code that
can be compiled in either environment.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Thu, Nov 14, 2013 at 09:33:59PM +1300, David Rowley wrote:
> On Sun, Nov 3, 2013 at 3:18 AM, David Rowley <dgrowleyml@gmail.com> wrote:
> > I'm low on ideas on how to improve things much around here for now, but
> > for what it's worth, I did create a patch which changes unnecessary calls
> > to appendPQExpBuffer() into calls to appendPQExpBufferStr() similar to the
> > recent one for appendStringInfo and appendStringInfoString.
> >
> Attached is a re-based version of this.

It does not apply anymore, could you resend it?

I am bit suspicious of performance impact of this patch, but think
that it's still worthwhile as it decreases code style where single
string argument is given to printf-style function without "%s".

-- 
marko

On Mon, Nov 18, 2013 at 06:18:01PM +1300, David Rowley wrote:
> On Mon, Nov 18, 2013 at 1:01 AM, Marko Kreen <markokr@gmail.com> wrote:
> > I am bit suspicious of performance impact of this patch, but think
> > that it's still worthwhile as it decreases code style where single
> > string argument is given to printf-style function without "%s".
> >
> >
> This thread probably did not explain very will the point of this patch.
> All this kicked up from an earlier patch which added for alignment in the
> log_line_prefix GUC. After some benchmarks were done on the proposed patch
> for that, it was discovered that replacing appendStringInfoString with
> appendStringInfo gave a big enough slowdown to matter in high volume
> logging scenarios. That patch was revised and the appendStringInfo()'s were
> only used when they were really needed and performance increased again.
> 
> I then ran a few benchmarks seen here:
> http://www.postgresql.org/message-id/CAApHDvp2uLKPuHJnCkonBGG2VXPvxoLOPzhrGXBS-M0r0v4wSA@mail.gmail.com
> 
> To compare appendStringInfo(si, "%s", str); with appendStringinfoString(a,
> str); and appendStringInfo(si, str);
> 
> The conclusion to those benchmarks were that appendStringInfoString() was
> the best function to use when no formatting was required, so I submitted a
> patch which replaced appendStringInfo() with appendStringInfoString() where
> that was possible and that was accepted.
> 
> appendPQExpBuffer() and appendPQExpBufferStr are the front end versions of
> appendStringInfo, so I spent an hour or so replacing these calls too as it
> should show a similar speedup, though in this case likely the performance
> is less critical, my thinking was more along the lines of, "it increases
> performance a little bit with a total of 0 increase in code complexity".

I was actually praising the patch that it reduces complexity,
so it's worth applying even if there is no performance win.

With performance win, it's doubleplus good.

The patch applies and regtests work fine.  So I mark it as
ready for committer.

> The findings in the benchmarks in the link above also showed that we might
> want to look into turning appendStringInfoString into a macro
> around appendBinaryStringInfo() to allow us to skip the strlen() call for
> string constants... It's unclear at the moment if this would be a good idea
> or much of an improvement, so it was left for something to think about for
> the future.

In any case it should be separate patch.  Perhaps applying the same
optimization for all such functions.

-- 
marko

On 18.11.2013 15:40, Marko Kreen wrote:
> On Mon, Nov 18, 2013 at 06:18:01PM +1300, David Rowley wrote:
>> On Mon, Nov 18, 2013 at 1:01 AM, Marko Kreen <markokr@gmail.com> wrote:
>>> I am bit suspicious of performance impact of this patch, but think
>>> that it's still worthwhile as it decreases code style where single
>>> string argument is given to printf-style function without "%s".
>>>
>>>
>> This thread probably did not explain very will the point of this patch.
>> All this kicked up from an earlier patch which added for alignment in the
>> log_line_prefix GUC. After some benchmarks were done on the proposed patch
>> for that, it was discovered that replacing appendStringInfoString with
>> appendStringInfo gave a big enough slowdown to matter in high volume
>> logging scenarios. That patch was revised and the appendStringInfo()'s were
>> only used when they were really needed and performance increased again.
>>
>> I then ran a few benchmarks seen here:
>> http://www.postgresql.org/message-id/CAApHDvp2uLKPuHJnCkonBGG2VXPvxoLOPzhrGXBS-M0r0v4wSA%40mail.gmail.com
>>
>> To compare appendStringInfo(si, "%s", str); with appendStringinfoString(a,
>> str); and appendStringInfo(si, str);
>>
>> The conclusion to those benchmarks were that appendStringInfoString() was
>> the best function to use when no formatting was required, so I submitted a
>> patch which replaced appendStringInfo() with appendStringInfoString() where
>> that was possible and that was accepted.
>>
>> appendPQExpBuffer() and appendPQExpBufferStr are the front end versions of
>> appendStringInfo, so I spent an hour or so replacing these calls too as it
>> should show a similar speedup, though in this case likely the performance
>> is less critical, my thinking was more along the lines of, "it increases
>> performance a little bit with a total of 0 increase in code complexity".
>
> I was actually praising the patch that it reduces complexity,
> so it's worth applying even if there is no performance win.
>
> With performance win, it's doubleplus good.
>
> The patch applies and regtests work fine.  So I mark it as
> ready for committer.

Ok, committed.

PS. I'm not 100% convinced that this kind of code churn is worthwhile. 
It doesn't make any difference to readability in my eyes, in general. In 
some cases it does, but in others it messes with indentation 
(describeOneTables in src/bin/psql/describe.c). It also makes 
backpatching more laborious. But it's done now, hopefully this is a 
one-off thing.

- Heikki