Merlin Moncure <mmoncure@gmail.com> writes:
> On Wed, Sep 8, 2010 at 4:03 PM, Sam Nelson <samn@consistentstate.com> wrote:
>> So ... yes, it seems that those four id's are somehow part of the problem.
>> They're on amazon EC2 boxes (yeah, we're not too fond of the EC2 boxes
>> either), so memtest isn't available, but no new corruption has cropped up
>> since they stopped killing the waiting queries (I just double checked - they
>> were getting corrupted rows constantly, and we haven't gotten one since that
>> script stopped killing queries).
> That's actually a startling indictment of ec2 -- how were you killing
> your queries exactly? You say this is repeatable? What's your
> setting of full_page_writes?
I think we'd established that they were doing kill -9 on backend
processes :-(. However, PG has a lot of track record that says that
backend crashes don't result in corrupt data. What seems more likely
to me is that the corruption is the result of some shortcut taken while
shutting down or migrating the ec2 instance, so that some writes that
Postgres thought got to disk didn't really.
regards, tom lane