If I'm reading the source code correctly, the PostgreSQL 7.3.4 JDBC
driver creates a ResultSet by evaluating the entire query, and putting
all the rows into a Vector. ResultSet.next() iterates over the
Vector. Is that in fact what is happening?
If so, this isn't very good for processing large ResultSets. The only
other open-source driver I could find is jxDBCon
(http://jxdbcon.sourceforge.net). Does anyone here have experience
with it?
One obvious application-level workaround is to add ORDER BY <primary
key> and LIMIT to the query, retrieve batches, keeping track of the
last primary key value, etc. That has lots of problems -- changing the
results of the overall scan (since each query runs at a different
time), and doing an index scan instead of a seq scan.
I'm considering modifying the driver, but that would put me behind on
my development schedule.
Jack Orenstein