Re: Critical performance problems on large databases

> Thanks, Tom, that's helpful! So, are you saying the the psql
> client does all the buffering and not the server? Is the
> buffering mandatory when using the libpq API? When you say
> "most frontend client side libraries" do you know of any
> exceptions who don't do that? Is the odbc client doing this
> too?

Yup, yup, and all the ones I know about but I do not know the details of
several (including jdbc and odbc).

> I can see myself fixing this problem, but would like to know
> if there is something else already?

Not that I know of.

I could imagine extending libpq reasonably straightforwardly:
assuming that you use PQsendQuery/PQgetResult, you could add a
PQgetPartialResult() that would extract the rows-received-so-far.
When it fails to get any more rows, a final PQgetResult would get the
query's success/fail status and possible error message.  Not sure about
all the details here; if you want to pursue it, start a discussion on
pgsql-interfaces.

            regards, tom lane