Re: C++, Postgres , libpqxx huge query

On 04 May 2014, at 10:57, alexandros_e <alexandros.ef@gmail.com> wrote:

> Hello experts. I have posted this question on stack overflow, but I did not
> get any detailed answer, so I thought I should cross post here. My
> apologies.
>
> I have to execute an SQL query to Postgres by the following code. The query
> returns a huge number of rows (40M or more) and has 4 integer fields: When I
> use a workstation with 32Gb everything works but on a 16Gb workstation the
> query is very slow (due to swapping I guess). Is there any way to tell the
> C++ to load rows at batches, without waiting the entire dataset? With Java I
> never had these issues before, due to the probably better JDBC driver.
>
> try {
>        work W(*Conn);
>        result r = W.exec(sql[sqlLoad]);
>        W.commit();
>
>        for (int rownum = 0; rownum < r.size(); ++rownum) {
>            const result::tuple row = r[rownum];
>            vid1 = row[0].as<int>();
>            vid2 = row[1].as<int>();
>            vid3 = row[2].as<int>();
>            .....
>
>    } catch (const std::exception &e) {
>        std::cerr << e.what() << std::endl;
>    }
>
> I am using PostgreSQL 9.3 and there I see this
> http://www.postgresql.org/docs/9.3/static/libpq-single-row-mode.html, but I
> do not how to use it on my C++ code. Your help will be appreciated.

I think the section of relevance is: http://www.postgresql.org/docs/9.3/static/libpq-async.html

Alban Hertroys
--
If you can't see the forest for the trees,
cut the trees and you'll find there is no forest.