Re: Practical impediment to supporting multiple SSL libraries

* Martijn van Oosterhout (kleptog@svana.org) wrote:
> On Thu, Apr 13, 2006 at 12:12:25PM +0100, Dave Page wrote:
> > > Ok. I'm not sure what this "double copying" you're referring
> > > to is,
> >
> > The libpq driver copies results out of the PGresult struct into the
> > internal QueryResult classes. With libpq out of the loop, data can go
> > straight from the wire into the QR.
>
> Hmm, the simplest improvement I can think of is one where you
> register a callback that libpq calls whenever it has received a new
> tuple.

You wouldn't want it on every tuple as that'd get expensive through
function calls.

> However, w.r.t. the copying, the pointers in get PGresult are in memory
> belonging to that result. As long as that PGresult hangs around, you
> should be able to just copy the pointers rather than the data? Or is
> this unacceptable?

It's actually pretty common (or seems to be anyway) to want to store the
data from the query result into your own data structure.  Yes, you could
just use pointers all over the place but that means you're going to have
to use things which understand PQresult everywhere as opposed to having a
generic 'storage manager' with other generic things (index creator,
aggregator, etc) which can be used with more than just PQresults.

> The only alternative I can think of is let users provide a callback
> that is given the number of bytes and it returns memory to store the
> data into. But that just seems unnecessarily complex, considering you
> could just copy the pointers.

You don't provide a callback, you have the user provide a memory region
to libpq which libpq can then fill in.  It's really not that difficult,
the API would really look quite a bit like PQexecParams, ie:

int PQgetTuples(PGresult *res,    // Returns number of tuples populated const int max_ntuples,    // Basically buffer
sizechar *result_set,        // Destination buffer const int *columnOffsets,    // integer array of offsets const int
*columnLengths,   // integer array of lengths, for checks const int record_len,        // Length of each structure int
*columnNulls,       // 0/1 for is not null / is null int resultFormat);        // Or maybe just binary? 

If we want to do conversion of the data in some way then we may need to
expand this to include that ability (but I don't think PQgetvalue does,
so...).

> > There are elements of the wire protocol that libpq doesn't actually
> > implement from what I recall. IIRC, they were added specifically for
> > JDBC but also intended to be used by psqlODBC as well. I forget the
> > details though as I wasn't so involved with the ODBC development back
> > then.
>
> Ugh, that's terrible. How do these features get tested if nothing
> within the main tree implements them.

I fully agree with this sentiment...

> > In addition of course, implementing the protocol natively does allow for
> > maximum flexibility.
>
> Maybe, but it should be possible to have a lot of flexibility without
> having many projects jump through all sorts of hoops everytime a new
> protocol version is created.

Indeed.
Thanks,
    Stephen