On Wednesday, April 06, 2016 10:33:16 AM Lars Arvidson wrote:
> > I'd guess it's probably more like option 3 - Glusterfs ate my database.
>
> Hi, thanks for your reply!
> We do archive logs on a distributed Glusterfs volume in case the streaming
> replication gets too far behind and the transaction logs have been removed.
> Would a restore of a corrupt archived log file give the symptoms we are
> seeing? Would not Postgresql detect that the logfile was corrupt? Are there
> some way I can analyze archived logs files to see if this is the problem?
>
If it's just storing the logs, I doubt it's the cause of the problem. You can
ignore my message. I had too much fun fighting with Gluster recently.
I reread your original full post, and the one thing that stuck out for me was
"the clusters are now replicating from each other". I feel like that could be
a problem. But someone more intimate with the replication might want to input
on that.
Other than that, I wonder if you just have a hardware problem with your
storage.