Обсуждение: Random disconnects
Hi All,
Just wondering whether someone might recognize the problem i am
having. First the intro: postgresql-8.1.0 running on FreeBSD
6.0-RELENG, on a dual xeon 3.0GHZ machine with 2gb memory/2gb swap.
The problem is we have been experiencing random disconnects from our
main application and a script. The application is a web spider
written in java that uses the postgresql db to store urls, and quite
a lot of them. The script is a perl script that coordinates several
of these crawlers, and has a low postgresql utilisation.
On the machines with the java app (there are two of them currently,
each with its own postgresql database) we use the latest jdbc:
postgresql-8.1-405.jdbc3.jar
So the errors we see are:
In java, this big fat exception:
org.postgresql.util.PSQLException: An I/O error occured while
sending to the backend.
Call stack:
org.postgresql.util.PSQLException: An I/O error occured while
sending to the backend.
at
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:214)
at
org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:437)
at
org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:353)
at
org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:307)
at
net.oneworld.issuemapper.CrawlToDBListener.dumpLinkToSpoolAndFreeUpMemory(CrawlToDBListener.java:214)
at
net.oneworld.issuemapper.CrawlToDBListener.crawled(CrawlToDBListener.java:165)
at websphinx.Crawler.sendLinkEvent(Crawler.java:1540)
at websphinx.Crawler.process(Crawler.java:1910)
at websphinx.Crawler.fetch(Crawler.java:1840)
at websphinx.Worm.run(Crawler.java:1994)
Caused by: java.net.SocketException: Network is unreachable
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:66)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:124)
at org.postgresql.core.PGStream.flush(PGStream.java:502)
at
org.postgresql.core.v3.QueryExecutorImpl.sendSync(QueryExecutorImpl.java:672)
at
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:187)
... 9 more
Where at CrawlToDBListener.java:214 we just execute a prepared
statement that inserts the url data, which goes allright 99.99999%
of the time.
The perl error we get is:
DBI connect('dbname=issuemaps_a;host=192.168.0.2','crawler',...)
failed: server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
at cbsd-lib.pl line 181
Can't call method "prepare" on an undefined value at cbsd-lib.pl
line 188, <SIZEFILEH> line 68.
Line 188 of cbsd-lib is a dbh->prepare line..
I guess the proper work-around is to wrap all database calls such
that when it detects a loss of connection, it will retry (which is
what i would've done in the first place if i was the one who had
written this software), or use dbh / jdbc persistent connections to
make this transparent.
But I think there might be some underlying problem and if so, i'd
like to know about it and fix it.
Gr,
Koen
--
K.F.J. Martens, Sonologic, http://www.sonologic.nl/
Networking, hosting, embedded systems, unix, artificial intelligence.
Public PGP key: http://www.metro.cx/pubkey-gmc.asc
Wondering about the funny attachment your mail program
can't read? Visit http://www.openpgp.org/
>In java, this big fat exception: > >org.postgresql.util.PSQLException: An I/O error occured while >sending to the backend. >Call stack: >org.postgresql.util.PSQLException: An I/O error occured while >sending to the backend. > at >org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:214) > at > > <snip> > at websphinx.Worm.run(Crawler.java:1994) >Caused by: java.net.SocketException: Network is unreachable > > <snip> >org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:187) > ... 9 more > > > >But I think there might be some underlying problem and if so, i'd >like to know about it and fix it. > > It seems to me that the JVM is having problems writing to the socket that connects to the postgres server with the "Network is unreachable" error being the cause. What's the connectivity from your clients to your database - localhost, same LAN, routed LANs, WAN, ...? Even though the perl script says "This probably means the server terminated abnormally before or while processing the request", I doubt that's what happened - more likely is you lost the network between them (but not if you were running on localhost, probably!). Cheers, Robin
Robin Iddon wrote: >> But I think there might be some underlying problem and if so, i'd >> like to know about it and fix it. >> >> > It seems to me that the JVM is having problems writing to the socket > that connects to the postgres server with the "Network is unreachable" > error being the cause. > What's the connectivity from your clients to your database - localhost, > same LAN, routed LANs, WAN, ...? > > Even though the perl script says "This probably means the server > terminated abnormally before or while processing the request", I doubt > that's what happened - more likely is you lost the network between them > (but not if you were running on localhost, probably!). Well, that is the strange thing, it is on the same machine. The client is in a freebsd jail however, the database is not. I suspected that maybe the default setting of 128 of maximum number of sockets had something to do with it, but increasing that (to, say, 32768) did not help much.. It is indeed not the case that the server exited abnormally, it is just fine. It could, of course, be that the server process serving the particular client died, then i should enable some more verbose logging probably to find out such things.. Something to dive into! Gr, Koen -- K.F.J. Martens, Sonologic, http://www.sonologic.nl/ Networking, hosting, embedded systems, unix, artificial intelligence. Public PGP key: http://www.metro.cx/pubkey-gmc.asc Wondering about the funny attachment your mail program can't read? Visit http://www.openpgp.org/