Обсуждение: Re: [BUGS] no way in LargeObject API to detect short read?
Anyone able to fix this? > Paul M. Aoki (aoki@acm.org) reports a bug with a severity of 3 > The lower the number the more severe it is. > > Short Description no way in LargeObject API to detect short > read? > > Long Description > org.postgresql.largeobject.LargeObject.read(byte[],int,int) > returns void instead of int. there's therefore no reliable, > non-exceptional way to detect short reads. > > the natural way to write blocked-read loops would be to assume > that read(byte[],int,int) returned the number of bytes read or > (e.g.) -1 on EOF. > > Sample Code > > > No file was uploaded with this report > > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania 19026
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> Anyone able to fix this?
here's a hack we've been using in-house (written by Jun Gabayan,
<jgabayan@pahv.xerox.com>).
you may not like the style but it's a stab at a solution.
--
Paul M. Aoki / Xerox Palo Alto Research Center / 3333 Coyote Hill Road
aoki@acm.org / Computer Science Laboratory / Palo Alto, CA 94304-1314
Index: LargeObject.java
===================================================================
RCS file: /project/placeless/cvsroot/placeless2/src/org/postgresql/largeobject/LargeObject.java,v
retrieving revision 1.1
retrieving revision 1.3
diff -r1.1 -r1.3
64c64,67
<
---
>
> private int pos = 0; //current position
> private int size = 0;
>
85a89,90
> pos = tell();
> size = size();
102a108
> if(fd == 0) return;
105a112
> fd = 0;
118a126,132
> // calculate available data to read to avoid reading pass the end
> // to avoid an exception
> pos = tell();
> int avail = size - pos;
> if(avail == 0) return null;
> if(avail < len) len = avail;
> try {
123c137,141
<
---
> }catch(SQLException se) {
> System.out.println("***LargeObject.read: Caught SQLException: " + se.getMessage());
> return null;
> }
>
157c175
< public void read(byte buf[],int off,int len) throws SQLException
---
> public int read(byte buf[],int off,int len) throws SQLException
159c177,180
< System.arraycopy(read(len),0,buf,off,len);
---
> byte mybuf[] = read(len);
> int sz = (mybuf != null) ? mybuf.length : -1; //must return -1 for end of data
> if(sz > 0) System.arraycopy(mybuf,0,buf,off,sz);
> return sz;
Hmmm, what's the performance issues with this? Is there going to be a problem
with very large LargeObject's?
Quoting "Paul M. Aoki" <aoki@acm.org>:
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > Anyone able to fix this?
>
> here's a hack we've been using in-house (written by Jun Gabayan,
> <jgabayan@pahv.xerox.com>).
>
> you may not like the style but it's a stab at a solution.
> --
> Paul M. Aoki / Xerox Palo Alto Research Center / 3333 Coyote Hill Road
> aoki@acm.org / Computer Science Laboratory / Palo Alto, CA
> 94304-1314
>
> Index: LargeObject.java
> ===================================================================
> RCS file:
> /project/placeless/cvsroot/placeless2/src/org/postgresql/largeobject/LargeObje
ct.java,v
> retrieving revision 1.1
> retrieving revision 1.3
> diff -r1.1 -r1.3
> 64c64,67
> <
> ---
> >
> > private int pos = 0; //current position
> > private int size = 0;
> >
> 85a89,90
> > pos = tell();
> > size = size();
> 102a108
> > if(fd == 0) return;
> 105a112
> > fd = 0;
> 118a126,132
> > // calculate available data to read to avoid reading pass the
> end
> > // to avoid an exception
> > pos = tell();
> > int avail = size - pos;
> > if(avail == 0) return null;
> > if(avail < len) len = avail;
> > try {
> 123c137,141
> <
> ---
> > }catch(SQLException se) {
> > System.out.println("***LargeObject.read: Caught
> SQLException: " + se.getMessage());
> > return null;
> > }
> >
> 157c175
> < public void read(byte buf[],int off,int len) throws SQLException
> ---
> > public int read(byte buf[],int off,int len) throws SQLException
> 159c177,180
> < System.arraycopy(read(len),0,buf,off,len);
> ---
> > byte mybuf[] = read(len);
> > int sz = (mybuf != null) ? mybuf.length : -1; //must return -1 for
> end of data
> > if(sz > 0) System.arraycopy(mybuf,0,buf,off,sz);
> > return sz;
>
--
Peter Mount peter@retep.org.uk
PostgreSQL JDBC Driver: http://www.retep.org.uk/postgres/
RetepPDF PDF library for Java: http://www.retep.org.uk/pdf/
Peter,
Given the current implementation of size(), I don't think this is a good
solution to the problem at hand. Since size() is an expensive call
(especially on large files), using it in this way wouldn't be a
performant solution. Using size() also requires additional round trips
to the database to get this information. There needs to be a better
solution that doesn't require the additional overhead.
thanks,
--Barry
Peter T Mount wrote:
>
> Hmmm, what's the performance issues with this? Is there going to be a problem
> with very large LargeObject's?
>
> Quoting "Paul M. Aoki" <aoki@acm.org>:
>
> > Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > > Anyone able to fix this?
> >
> > here's a hack we've been using in-house (written by Jun Gabayan,
> > <jgabayan@pahv.xerox.com>).
> >
> > you may not like the style but it's a stab at a solution.
> > --
> > Paul M. Aoki / Xerox Palo Alto Research Center / 3333 Coyote Hill Road
> > aoki@acm.org / Computer Science Laboratory / Palo Alto, CA
> > 94304-1314
> >
> > Index: LargeObject.java
> > ===================================================================
> > RCS file:
> > /project/placeless/cvsroot/placeless2/src/org/postgresql/largeobject/LargeObje
> ct.java,v
> > retrieving revision 1.1
> > retrieving revision 1.3
> > diff -r1.1 -r1.3
> > 64c64,67
> > <
> > ---
> > >
> > > private int pos = 0; //current position
> > > private int size = 0;
> > >
> > 85a89,90
> > > pos = tell();
> > > size = size();
> > 102a108
> > > if(fd == 0) return;
> > 105a112
> > > fd = 0;
> > 118a126,132
> > > // calculate available data to read to avoid reading pass the
> > end
> > > // to avoid an exception
> > > pos = tell();
> > > int avail = size - pos;
> > > if(avail == 0) return null;
> > > if(avail < len) len = avail;
> > > try {
> > 123c137,141
> > <
> > ---
> > > }catch(SQLException se) {
> > > System.out.println("***LargeObject.read: Caught
> > SQLException: " + se.getMessage());
> > > return null;
> > > }
> > >
> > 157c175
> > < public void read(byte buf[],int off,int len) throws SQLException
> > ---
> > > public int read(byte buf[],int off,int len) throws SQLException
> > 159c177,180
> > < System.arraycopy(read(len),0,buf,off,len);
> > ---
> > > byte mybuf[] = read(len);
> > > int sz = (mybuf != null) ? mybuf.length : -1; //must return -1 for
> > end of data
> > > if(sz > 0) System.arraycopy(mybuf,0,buf,off,sz);
> > > return sz;
> >
>
> --
> Peter Mount peter@retep.org.uk
> PostgreSQL JDBC Driver: http://www.retep.org.uk/postgres/
> RetepPDF PDF library for Java: http://www.retep.org.uk/pdf/
Peter T Mount <peter@retep.org.uk> writes: > Hmmm, what's the performance issues with this? Is there going to be a problem > with very large LargeObject's? you could probably be smarter about caching previous tell() state, if that's what you mean. jun's hack doesn't actually add any extra buffers or copies. -- Paul M. Aoki / Xerox Palo Alto Research Center / 3333 Coyote Hill Road aoki@acm.org / Computer Science Laboratory / Palo Alto, CA 94304-1314
At 15:37 26/01/01 -0800, Barry Lind wrote:
>Peter,
>
>Given the current implementation of size(), I don't think this is a good
>solution to the problem at hand. Since size() is an expensive call
>(especially on large files), using it in this way wouldn't be a
>performant solution. Using size() also requires additional round trips
>to the database to get this information. There needs to be a better
>solution that doesn't require the additional overhead.
That's exactly what I was thinking of. Most of the largeobject API is based
on libpq hence where this came from. I don't think there is
anything available elsewhere for getting the size.
Yes, there needs to be a better way.
Peter
>thanks,
>--Barry
>
>Peter T Mount wrote:
> >
> > Hmmm, what's the performance issues with this? Is there going to be a
> problem
> > with very large LargeObject's?
> >
> > Quoting "Paul M. Aoki" <aoki@acm.org>:
> >
> > > Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > > > Anyone able to fix this?
> > >
> > > here's a hack we've been using in-house (written by Jun Gabayan,
> > > <jgabayan@pahv.xerox.com>).
> > >
> > > you may not like the style but it's a stab at a solution.
> > > --
> > > Paul M. Aoki / Xerox Palo Alto Research Center / 3333 Coyote Hill Road
> > > aoki@acm.org / Computer Science Laboratory / Palo Alto, CA
> > > 94304-1314
> > >
> > > Index: LargeObject.java
> > > ===================================================================
> > > RCS file:
> > >
> /project/placeless/cvsroot/placeless2/src/org/postgresql/largeobject/LargeObje
> > ct.java,v
> > > retrieving revision 1.1
> > > retrieving revision 1.3
> > > diff -r1.1 -r1.3
> > > 64c64,67
> > > <
> > > ---
> > > >
> > > > private int pos = 0; //current position
> > > > private int size = 0;
> > > >
> > > 85a89,90
> > > > pos = tell();
> > > > size = size();
> > > 102a108
> > > > if(fd == 0) return;
> > > 105a112
> > > > fd = 0;
> > > 118a126,132
> > > > // calculate available data to read to avoid reading pass the
> > > end
> > > > // to avoid an exception
> > > > pos = tell();
> > > > int avail = size - pos;
> > > > if(avail == 0) return null;
> > > > if(avail < len) len = avail;
> > > > try {
> > > 123c137,141
> > > <
> > > ---
> > > > }catch(SQLException se) {
> > > > System.out.println("***LargeObject.read: Caught
> > > SQLException: " + se.getMessage());
> > > > return null;
> > > > }
> > > >
> > > 157c175
> > > < public void read(byte buf[],int off,int len) throws SQLException
> > > ---
> > > > public int read(byte buf[],int off,int len) throws SQLException
> > > 159c177,180
> > > < System.arraycopy(read(len),0,buf,off,len);
> > > ---
> > > > byte mybuf[] = read(len);
> > > > int sz = (mybuf != null) ? mybuf.length : -1; //must return -1 for
> > > end of data
> > > > if(sz > 0) System.arraycopy(mybuf,0,buf,off,sz);
> > > > return sz;
> > >
> >
> > --
> > Peter Mount peter@retep.org.uk
> > PostgreSQL JDBC Driver: http://www.retep.org.uk/postgres/
> > RetepPDF PDF library for Java: http://www.retep.org.uk/pdf/