[HACKERS] libpq: why we need to fetch all rows?

Hello!

  Currently libpq works as follows:

  If we doing a select, all rows fetched in memory, after this
  it returned to the caller.

  All OK, but: if number of rows is too big, we use a LOT of memory.

  And, if select scans large table, we get first result after long
  (relatively ;) time...

  Many SQL servers will return rows as soon as it found (ASAF), and this
  behavior is more friendly - human don't need to wait before
  first result is appears.

  Just imagine typical query from WEB: large database,
  and select with condition which will take long time.

  In my case, I've database with >150,000 rows, and often query
  is based on RE, so search will take long time. For end user,
  this looks like server hang, at least if he used it first time.

  Of course, implementing ASAF will require some changes in programs
  which use libpq, and will cost something - e.g., program will not
  know total number of rows returned (does anybody use row count?).

  But at least this functionality must be as option, IMHO.

  Especially, when using psql in batch mode, nobody is interested
  in total row count, since: 1) it returned after last row;
  2) after last row is fetched from psql output, we already know
  total row count. ;)

  I've modified libpq for my purposes, but what do you think about
  adding this to future versions? May be, as alternative for
  currently distributed libpq...

/Al

------------------------------