Re: [patch] libpq one-row-at-a-time API

On Tue, Jul 24, 2012 at 11:39 AM, Marko Kreen <markokr@gmail.com> wrote:
> On Tue, Jul 24, 2012 at 7:25 PM, Merlin Moncure <mmoncure@gmail.com> wrote:
>> On Tue, Jul 24, 2012 at 11:08 AM, Marko Kreen <markokr@gmail.com> wrote:
>>>> I'm arguing that *all* data getting must continue to do so through the
>>>> result object, and bypassing the result to get at data is breaking the
>>>> result abstraction in the libpq api.  I bet you can still maintain
>>>> data access through result object while avoiding extra copies.
>>>
>>> Well, the main problem this week is whether we should
>>> apply PQsetSingleRowMode() from single-row-mode1
>>> or from single-row-mode2 branch?
>>>
>>> The PQgetRowData() is unimportant as it just exposes
>>> the rowBuf to user and thats all.
>>
>> right. branch 1 (containing PQgetRowData) seems wrong to me.
>
> Indeed, you are missing the fact that PGgetResult works same
> in both branches.,
>
> And please see the benchmart I posted.
>
> Even better, run it yourself...
>
>>> What do you mean by that?  And have you though about both
>>> sync and async usage patterns?
>>
>> No, I haven't -- at least not very well.  The basic idea is that
>> PQsetSingleRowMode turns into PQgetSingleRowResult() and returns a
>> result object.  For row by row an extra API call gets called (maybe
>> PQgetNextRow(PGconn, PGresult)) that does the behind the scenes work
>> under the existing result object.  This is the same general structure
>> you have in branch 2, but the result allocation is move out of the
>> loop.  Presumably sync and async would then follow the same pattern,
>> but we'd still have to be able to guarantee non-blocking behavior in
>> the async api.
>
> Well, as discussed previously it's worthwhile to keep standard functions
> like PQisBusy() and PQgetResult() working sensibly in new mode,
> and currently they do so.
>
> If you add new functions, you also need to define the behavior
> when new and old one's are mixed and it gets messy quickly.

Right, I'm aware that you can use 'slow' method in branch 1.  I also
saw the benchmarks and agree that single row mode should be at least
competitive with current methods for pretty much all cases.

But, the faster rowbuf method is a generally incompatible way of
dealing with data vs current libpq -- this is bad.  If it's truly
impossible to get those benefits without bypassing result API that
then I remove my objection on the grounds it's optional behavior (my
gut tells me it is possible though).

I think the dummy copy of PGresult is plausible (if by that you mean
optimizing PQgetResult when in single row mode).  That would be even
better: you'd remove the need for the rowbuf mode.

merlin