Hi,
Recently two of our database servers crashed and won't start. The errors
in postgres logs are:
Server 1:
2017-07-16 11:36:12 UTC PANIC: could not access status of transaction
392483501
2017-07-16 11:36:12 UTC DETAIL: Could not read from file
"pg_multixact/members/14077" at offset 90112: Success.
"pg_multixact/members" contains files: D4DD-FFFF, 10000-14077, 0000
"pg_multixact/offsets" contains files: 169E-1764
Server 2:
2017-07-17 00:49:48 UTC PANIC: could not access status of transaction
1433865475
2017-07-17 00:49:48 UTC DETAIL: Could not open file
"pg_multixact/members/14078": No such file or directory.
"pg_multixact/members" contains files: F10D-FFFF, 10000-14078, 0000
"pg_multixact/offsets" contains files: 5366-5577
After the error, postgres terminated all its processes, restarted, tried
to recover, but failed with similar error.
We're running postgres 9.3.4 which had bugs related to "multixact
wraparound" (fixed in following versions), and according to the
following link, the numbers 14077/14078 are at/near the multixact
"members" wraparound point:
https://www.postgresql.org/message-id/20131209182701.GD9519%40awork2.anarazel.de
My understanding is that due to that bug the transaction logs are now
corrupt. We do have some backups, but we would like to try to recover
the more recent transactions, at least up to those which were committed
during the last checkpoint.
Did anyone else also have that kind of issue?
How can we recover the more recent transactions? Should we simply reset
the corrupt transaction logs (using postgres tools), or are there other
things we could try? Thanks!
Regards,
Vitaliy