I have a Ruby on Rails web site that is running on a CentOS 5 server using Apache 2 with Passenger and PostgreSQL 8.4. Over the past two weeks I am seeing my PostgreSQL daemon, postfix, and rsyslog daemons all die at the same time, this happens at least 5 times per day at random times. Apache however, does not die.
I cannot associate it with any specific URL on the site or database query, it doesn’t match up with any scheduled tasks, and hardly reports anything to most logs. The only relevant log entry I can find is below from my PostgreSQL logs. I’ve searched through many PostgreSQL list archives and found little information. Only the suggestion that something might be sending SIGTERM commands to PostgreSQL or that system limits may be reached. The server has plenty of free disk space and 1 GB of ram with only 250MB in use. It’s a dedicated web server with no other processes running really, just the base CentOS processes and the ones I mentioned at the start of this post.
It seems to be the result of database query within my Ruby code. The processes do not die when apache isn’t running, but I cannot pin point it. I’ve even rebuilt the server from scratch and am having the same results. Has anyone seen anything like this before?
2010-05-19 14:40:45 MST | 127.0.0.1FATAL: terminating connection due to administrator command
2010-05-19 14:40:45 MST | 127.0.0.1DEBUG: shmem_exit(1): 6 callbacks to make
2010-05-19 14:40:45 MST | 127.0.0.1DEBUG: proc_exit(1): 3 callbacks to make
2010-05-19 14:40:45 MST | 127.0.0.1DEBUG: exit(1)
2010-05-19 14:40:45 MST | 127.0.0.1LOG: could not send data to client: Bad file descriptor
2010-05-19 14:40:45 MST | 127.0.0.1DEBUG: shmem_exit(-1): 0 callbacks to make
2010-05-19 14:40:45 MST | 127.0.0.1DEBUG: proc_exit(-1): 0 callbacks to make
Thanks for any help,
SpencerPostgreSQL database and other daemons crashing