Sam Nelson <samn@consistentstate.com> writes:
>> It's almost certainly not ruby's fault. Have they done anything
>> strange like kill the instance and restart it without letting the db
>> shut down? I'd tend to suspect Amazon's fsyncing is amiss and they
>> did something that triggered it.
> They haven't done anything like that, that we know of. However, they do
> have a process that kills off all waiting (and only waiting) postgres
> processes if there are more than 1000 locks. Could that be an issue?
What do you mean by "kills off"? If they randomly kill -9 backend
processes, I would describe that as taking pot-shots at one's own toes
(and hoping that you can't aim well enough to hit them). In theory
Postgres should survive that without data corruption but it's surely
playing with fire. And it's most definitely not solving whatever their
real problem is.
Anyway, the known issues in this area have to do with the filesystem not
honoring writes in the correct order. I'd agree with Scott's suspicion
that a dirty instance shutdown, rather than a dirty database shutdown,
is the more likely cause.
regards, tom lane