On Tue, May 27, 2014 at 11:06 AM, Heikki Linnakangas <
hlinnakangas@vmware.com> wrote:
> I would be interested in seeing the structure of the index, if there is
> anything else corrupt in there.
It's an index on (integer, timestamp without time zone). Unfortunately,
it's a customer DB, so getting more direct access may be problematic. Is
there metadata we can gather about it that could be useful?
Also, what WAL actions led to the error? Try something like:
>
> pg_xlogdump -r btree -p $PGDATA -s 339/65000000 | grep 1665279
>
> and search that for any records related to the failed split, e.g. grepping
> further for the block numbers in the error message.
That gives me no output even without the grep (except to complain that the
next segment is missing unless I pass '-e 339/65FFFFFF', which is
reasonable, right?). I also had to change the timeline with `-t 2`, but I
don't imagine that makes a difference here. If I go back further, I do get
output for that index, but nothing that mentions 175193 or 193740.
Also, it turned out that this was a persistent problem--a couple of
replicas failed the same way. I then worked with the customer and had them
re-create the index, and that seems to have resolved the issue. My
colleague Greg Stark has taken over the forensic investigation here--he may
have more to add.