Re: BlobOutputStream patch and related patches

Patch applied.

--Barry


David Wall wrote:
> Not sure if this was done, Barry, or not.  But as you mentioned, the output
> stream didn't override the byte array method, so this is a diff that shows
> that bit added so that buffers can be written.  I was able to do basic tests
> of these under 7.2.2.
>
> This was all done on the original 7.2.2 sources (the .orig version in the
> diff).
>
>
> This includes the new BlobOutputStream.write() of a buffer, including the
> performance boost of saving the buffer creation/copy in LargeObject when the
> buffer being written is in its entirety.
>
> [postgresql@dev1 largeobject]$ diff -c BlobOutputStream.orig
> BlobOutputStream.java
> *** BlobOutputStream.orig       Mon Nov 19 14:33:39 2001
> --- BlobOutputStream.java       Mon Sep  2 10:58:03 2002
> ***************
> *** 68,73 ****
> --- 68,92 ----
>                 }
>         }
>
> +       public void write(byte[] buf, int off, int len) throws
> java.io.IOException
> +       {
> +               try
> +               {
> +                   // If we have any internally buffered data, send it
> first
> +                   if ( bpos > 0 )
> +                       flush();
> +
> +                   if ( off == 0 && len == buf.length )
> +                       lo.write(buf); // save a buffer creation and copy
> since full buffer written
> +                   else
> +                       lo.write(buf,off,len);
> +               }
> +               catch (SQLException se)
> +               {
> +                       throw new IOException(se.toString());
> +               }
> +       }
> +
>
>
> This shows the new buffered writing patch for PreparedStatement.setBlob().
> Note that Barry said this patch was applied, but there is the additional fix
> to save the LargeObject buffer creations in the main loop (the test if the
> numRead == buf.length) where if the buffer is full, a different call is used
> than when it's not full, saving a buffer creation and copy in the underlying
> writes.  This is true for all of the Blob output except for the "last"
> buffer which is only full if the Blob itself was a multiple of 4k.
>
> [postgresql@dev1 jdbc2]$ diff -c PreparedStatement.java
> PreparedStatement.orig
> *** PreparedStatement.java      Mon Sep  2 10:58:27 2002
> --- PreparedStatement.orig      Mon Jan 14 23:37:33 2002
> ***************
> *** 879,918 ****
>         public void setBlob(int i, Blob x) throws SQLException
>         {
>                 InputStream l_inStream = x.getBinaryStream();
>                 LargeObjectManager lom = connection.getLargeObjectAPI();
>                 int oid = lom.create();
>                 LargeObject lob = lom.open(oid);
>                 OutputStream los = lob.getOutputStream();
> -               byte[] buf = new byte[4096];
>                 try
>                 {
>                         // could be buffered, but then the OutputStream
> returned by LargeObject
>                         // is buffered internally anyhow, so there would be
> no performance
>                         // boost gained, if anything it would be worse!
> !                       int bytesRemaining = (int)x.length();
> !                       int numRead =
> l_inStream.read(buf,0,Math.min(buf.length,bytesRemaining));
> !                       while (numRead != -1 && bytesRemaining > 0)
>                         {
> !                               bytesRemaining -= numRead;
> !                               if ( numRead == buf.length )
> !                                   los.write(buf); // saves a buffer
> creation and copy in LargeObject since it's full
> !                               else
> !                                   los.write(buf,0,numRead);
> !                               numRead =
> l_inStream.read(buf,0,Math.min(buf.length,bytesRemaining));
>                         }
>                 }
>                 catch (IOException se)
>                 {
>                         throw new PSQLException("postgresql.unusual", se);
> -               }
> -               finally
> -               {
> -                   try
> -                   {
> -                           los.close();
> -                 l_inStream.close();
> -             }
> -             catch( Exception e ) {}
>                 }
>                 // lob is closed by the stream so don't call lob.close()
>                 setInt(i, oid);
> --- 879,907 ----
>         public void setBlob(int i, Blob x) throws SQLException
>         {
>                 InputStream l_inStream = x.getBinaryStream();
> +               int l_length = (int) x.length();
>                 LargeObjectManager lom = connection.getLargeObjectAPI();
>                 int oid = lom.create();
>                 LargeObject lob = lom.open(oid);
>                 OutputStream los = lob.getOutputStream();
>                 try
>                 {
>                         // could be buffered, but then the OutputStream
> returned by LargeObject
>                         // is buffered internally anyhow, so there would be
> no performance
>                         // boost gained, if anything it would be worse!
> !                       int c = l_inStream.read();
> !                       int p = 0;
> !                       while (c > -1 && p < l_length)
>                         {
> !                               los.write(c);
> !                               c = l_inStream.read();
> !                               p++;
>                         }
> +                       los.close();
>                 }
>                 catch (IOException se)
>                 {
>                         throw new PSQLException("postgresql.unusual", se);
>                 }
>                 // lob is closed by the stream so don't call lob.close()
>                 setInt(i, oid);
>
>
>
>
>
>
>
>
>
> This change was made so that the LargeObject calls use the same 4k buffer
> size as used in in setBlob() for input/output streams.
>
>
> [postgresql@dev1 largeobject]$ diff -c LargeObject.orig LargeObject.java
> *** LargeObject.orig    Mon Nov 19 14:33:39 2001
> --- LargeObject.java    Mon Sep  2 10:58:10 2002
> ***************
> *** 299,305 ****
>          */
>         public InputStream getInputStream() throws SQLException
>         {
> !               return new BlobInputStream(this);
>         }
>
>         /*
> --- 299,305 ----
>          */
>         public InputStream getInputStream() throws SQLException
>         {
> !               return new BlobInputStream(this,4096);
>         }
>
>         /*
> ***************
> *** 313,319 ****
>         public OutputStream getOutputStream() throws SQLException
>         {
>                 if (os == null)
> !                       os = new BlobOutputStream(this);
>                 return os;
>         }
>
> --- 313,319 ----
>         public OutputStream getOutputStream() throws SQLException
>         {
>                 if (os == null)
> !                       os = new BlobOutputStream(this,4096);
>                 return os;
>         }
>
>
>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster
>