Обсуждение: table full scan
Hi, I am using jdbc to full scan a huge table (over 11 million records). Unfortunately, I get an OutOfMemoryError exception. Is there any way to tell postgre not to try to load the whole table at once? Best Regards. Khaldoun
On Thu, 5 Jan 2006, Khaldoun Ateyeh wrote: > I am using jdbc to full scan a huge table (over 11 million records). > Unfortunately, I get an OutOfMemoryError exception. Is there any way to > tell postgre not to try to load the whole table at once? > http://jdbc.postgresql.org/documentation/81/query.html#query-with-cursor Kris Jurka
"Khaldoun Ateyeh" <Khaldoun.Ateyeh@panoratio.de> writes: > Hi, > I am using jdbc to full scan a huge table (over 11 million records). > Unfortunately, I get an OutOfMemoryError exception. Is there any way to > tell postgre not to try to load the whole table at once? > Set autocommit to false, then use the method setFetchSize() of the Statement to set the maximal rows that are read into the memory. For example: con.setAutoCommit(false); stmt = con.createStatement(); stmt.setFetchSize(1000); -- Roland Walter mailto: rwa (at) mosaic-ag (dot) com MOSAIC SOFTWARE AG phone: +49 (0) 22 25 / 88 2-41 1 Am Pannacker 3 fax: +49 (0) 22 25 / 88 2-20 1 D-53340 Meckenheim http://www.mosaic-ag.com Die in dieser E-Mail enthaltenen Nachrichten und Anhaenge sind ausschliesslich fuer den bezeichneten Adressaten bestimmt. Sie koennen rechtlich geschuetzte, vertrauliche Informationen enthalten. Falls Sie nicht der bezeichnete Empfaenger oder zum Empfang dieser E-Mail nicht berechtigt sind, ist die Verwendung, Vervielfaeltigung oder Weitergabe von Nachrichten und Anhaengen untersagt. Falls Sie diese E-Mail irrtuemlich erhalten haben, informieren Sie bitte unverzueglich den Absender und vernichten Sie die E-Mail. This e-mail message and any attachment are intended exclusively for the named addressee. They may contain confidential information which may also be protected by professional secrecy. Unless you are the named addressee (or authorised to receive for the addressee) you may not copy or use this message or any attachment or disclose the contents to anyone else. If this e-mail was sent to you by mistake please notify the sender immediately and delete this e-mail.
Hi, Which driver should I use? Because when I do what you suggested, I get "This method is not yet implemented" ... stmt.setFetchSize(1000). I am using the official postgresql-8.1-404.jdbc3.jar driver. Many thanks. Khaldoun -----Original Message----- From: Roland Walter [mailto:rwa@mosaic-ag.com] Sent: Thursday, January 05, 2006 1:15 PM To: Khaldoun Ateyeh Cc: pgsql-jdbc@postgresql.org Subject: Re: [JDBC] table full scan "Khaldoun Ateyeh" <Khaldoun.Ateyeh@panoratio.de> writes: > Hi, > I am using jdbc to full scan a huge table (over 11 million records). > Unfortunately, I get an OutOfMemoryError exception. Is there any way to > tell postgre not to try to load the whole table at once? > Set autocommit to false, then use the method setFetchSize() of the Statement to set the maximal rows that are read into the memory. For example: con.setAutoCommit(false); stmt = con.createStatement(); stmt.setFetchSize(1000); -- Roland Walter mailto: rwa (at) mosaic-ag (dot) com MOSAIC SOFTWARE AG phone: +49 (0) 22 25 / 88 2-41 1 Am Pannacker 3 fax: +49 (0) 22 25 / 88 2-20 1 D-53340 Meckenheim http://www.mosaic-ag.com
On Thu, 5 Jan 2006, Khaldoun Ateyeh wrote: > Which driver should I use? Because when I do what you suggested, I get > "This method is not yet implemented" ... stmt.setFetchSize(1000). > > I am using the official postgresql-8.1-404.jdbc3.jar driver. > Apparently not. You most likely have a 7.3 or earlier driver in your classpath as well because this method has been implemented for quite some time, and certainly is in 8.1. Kris Jurka
You are right! I had another older version driver in the classpath. Thanks for your help. Khaldoun -----Original Message----- From: Kris Jurka [mailto:books@ejurka.com] Sent: Thursday, January 05, 2006 2:07 PM To: Khaldoun Ateyeh Cc: Roland Walter; pgsql-jdbc@postgresql.org Subject: Re: [JDBC] table full scan On Thu, 5 Jan 2006, Khaldoun Ateyeh wrote: > Which driver should I use? Because when I do what you suggested, I get > "This method is not yet implemented" ... stmt.setFetchSize(1000). > > I am using the official postgresql-8.1-404.jdbc3.jar driver. > Apparently not. You most likely have a 7.3 or earlier driver in your classpath as well because this method has been implemented for quite some time, and certainly is in 8.1. Kris Jurka