Обсуждение: Using bytea with ResultSet.getBytes("..."). Problem. Help!!

Поиск
Список
Период
Сортировка

Using bytea with ResultSet.getBytes("..."). Problem. Help!!

От
"roman"
Дата:
Hello!
(I use Netbeans 5.5, Jdk 1.6, postgreSql 8.1.5 on linux host).
 
 
I have a problem with reading big binary data from BYTEA field.
I write to it about 16MB pdf file (as byte[] array) sucessfully, but when I try to read it back I have an exception:
 
Exception occurred during event dispatching:
java.lang.OutOfMemoryError: Java heap space
        at org.postgresql.util.PGbytea.toBytes(PGbytea.java:53)
        at org.postgresql.jdbc2.AbstractJdbc2ResultSet.getBytes(AbstractJdbc2ResultSet.java:2152)
        at org.postgresql.jdbc2.AbstractJdbc2ResultSet.getBinaryStream(AbstractJdbc2ResultSet.java:2272)
        at org.postgresql.jdbc2.AbstractJdbc2ResultSet.getBinaryStream(AbstractJdbc2ResultSet.java:2367)
I have no problem with files about 1MB.
 
My code for writing object:
...
    java.sql.PreparedStatement insAttachment=connection.prepareStatement("INSERT INTO \"T_Table\" (\"f_InitialDocument\") VALUES (?)"); //f_InitialDocument is a BYTEA field
...
    public void insertAttachment(String FILENAME) {
        byte[] buf=null;
        try {
            //first read file to buf
            java.io.File f=new java.io.File(FILENAME);
            if(f.exists() && f.canRead()) {
                java.io.RandomAccessFile raf=new java.io.RandomAccessFile(f, "r");
                buf=new byte[(int)raf.length()];
                raf.readFully(buf);
                raf.close();
                //Write it to database
                insAttachment.setBytes(1, buf);
                insAttachment.execute();
            } else { ret=false; }
        } catch(Exception e) {
            ret=false;
        }
    }
My code for reading object:
...
     String query="SELECT \"f_InitialDocument\" FROM \"T_Attachments\"";
     java.sql.ResultSet rs=database.executeFastQuery(query); //My function for executing query which gives java.sql.ResultSet
     if(rs!=null) {
         try {
             if(rs.next()) {
                 byte[] buf=rs.getBytes("f_InitialDocument"); //HERE I GOT THAT EXCEPTION ABOUT OutOfMemory!!!
                 //Create a template file
                 String filename=database.getTempFilename(); //Function for generating temporary filenames basing on GUID
                  java.io.File f=new java.io.File(filename);
                  if(f.createNewFile()) {
                      java.io.FileOutputStream fos=new java.io.FileOutputStream(f);
                      fos.write(buf);
                      fos.close();
                      database.executeCommand(filename); //function for opening file in appropiate application
                  }
             }
         } catch(Exception e){}
     }
...
 
I tried to launch my program with options: -Xmx800M -Xms1500M, but it didn't help...
 
I tried to use: java.io.InputStream is=rs.getBinaryStream("f_InitialDocument"); but got the same error in this string in debugger.
 
How can I read such big objects???
Please help me.
 
Roman.
 
 
 

Re: Using bytea with ResultSet.getBytes("..."). Problem. Help!!

От
Dave Cramer
Дата:

On 19-Jul-07, at 12:20 AM, roman wrote:

Hello!
(I use Netbeans 5.5, Jdk 1.6, postgreSql 8.1.5 on linux host).
 
 
I have a problem with reading big binary data from BYTEA field.
I write to it about 16MB pdf file (as byte[] array) sucessfully, but when I try to read it back I have an exception:
 
Exception occurred during event dispatching:
java.lang.OutOfMemoryError: Java heap space
        at org.postgresql.util.PGbytea.toBytes(PGbytea.java:53)
        at org.postgresql.jdbc2.AbstractJdbc2ResultSet.getBytes(AbstractJdbc2ResultSet.java:2152)
        at org.postgresql.jdbc2.AbstractJdbc2ResultSet.getBinaryStream(AbstractJdbc2ResultSet.java:2272)
        at org.postgresql.jdbc2.AbstractJdbc2ResultSet.getBinaryStream(AbstractJdbc2ResultSet.java:2367)
I have no problem with files about 1MB.

The driver has to take it all into memory, so if you don't have enough memory for the file, you will run out of memory.
 
My code for writing object:
...
    java.sql.PreparedStatement insAttachment=connection.prepareStatement("INSERT INTO \"T_Table\" (\"f_InitialDocument\") VALUES (?)"); //f_InitialDocument is a BYTEA field
...
    public void insertAttachment(String FILENAME) {
        byte[] buf=null;
        try {
            //first read file to buf
            java.io.File f=new java.io.File(FILENAME);
            if(f.exists() && f.canRead()) {
                java.io.RandomAccessFile raf=new java.io.RandomAccessFile(f, "r");
                buf=new byte[(int)raf.length()];
                raf.readFully(buf);
                raf.close();
                //Write it to database
                insAttachment.setBytes(1, buf);
                insAttachment.execute();
            } else { ret=false; }
        } catch(Exception e) {
            ret=false;
        }
    }
My code for reading object:
...
     String query="SELECT \"f_InitialDocument\" FROM \"T_Attachments\"";
     java.sql.ResultSet rs=database.executeFastQuery(query); //My function for executing query which gives java.sql.ResultSet
     if(rs!=null) {
         try {
             if(rs.next()) {
                 byte[] buf=rs.getBytes("f_InitialDocument"); //HERE I GOT THAT EXCEPTION ABOUT OutOfMemory!!!
                 //Create a template file
                 String filename=database.getTempFilename(); //Function for generating temporary filenames basing on GUID
                  java.io.File f=new java.io.File(filename);
                  if(f.createNewFile()) {
                      java.io.FileOutputStream fos=new java.io.FileOutputStream(f);
                      fos.write(buf);
                      fos.close();
                      database.executeCommand(filename); //function for opening file in appropiate application
                  }
             }
         } catch(Exception e){}
     }
...
 
I tried to launch my program with options: -Xmx800M -Xms1500M, but it didn't help...
 
I tried to use: java.io.InputStream is=rs.getBinaryStream("f_InitialDocument"); but got the same error in this string in debugger.
 
How can I read such big objects???
Please help me.
 
Roman.
 
 
 


Re: Using bytea with ResultSet.getBytes("..."). Problem. Help!!

От
Kris Jurka
Дата:

On Thu, 19 Jul 2007, roman wrote:

> I have a problem with reading big binary data from BYTEA field. I write
> to it about 16MB pdf file (as byte[] array) sucessfully, but when I try
> to read it back I have an exception:
>
> Exception occurred during event dispatching:
> java.lang.OutOfMemoryError: Java heap space
>        at org.postgresql.util.PGbytea.toBytes(PGbytea.java:53)

You don't mention what driver version you are using, but 8.2-504 and
higher will be slightly stingier about the amount of memory it uses.  You
still need a lot of memory to retrieve large results, but this version is
better.

Kris Jurka