On Dec 07 06:01, Alexander Scholz wrote:
> I have a serious performance flaw, when using postgresql 8.1 (other
> versions haven't been tested) with libpq. When executing a
>
> select * from "xyz" 1)
>
> and "xyz" contains 300'000 records, it takes more than 60 seconds for
> the query just to complete. First I thought it is a performance problem
> of the server, but then we noticed an enormous amount of TCP/IP packets
> that are beeing received in that (waiting)time. The memory usage of my
> process, while waiting for the query to complete, bumps up to more than
> 300MB, the CPU usage goes up to 100%! It seems as if libpq would be
> receiving the complete resultset from the server _before_ returning from
> the query execution! (Even then 300MB seems for me to be too much, but
> that's not the point!) Just to ensure: I didn't perform ANY _fetch_ so
> far! It's just about executing the query.
libpq just works as expected. The query you sent returned 300 megs of data
and libpq read returned data from the socket then stored it in the memory.
There is nothing special in this behaviour. What would you expect? To
PostgreSQL hold 300 megs of data waiting for you in the server side memory?
As Alexander Scholz said, use CURSORs to achieve a functionality as you
expected.
If you're interested in queries returning incremental results, you can
take a look at mailing-list archives for related topic. Furthermore,
there's a TODO item for this too.
Regards.
--
"We are the middle children of history, raised by television to believe
that someday we'll be millionaires and movie stars and rock stars, but
we won't. And we're just learning this fact," Tyler said. "So don't
fuck with us."