Alvaro Herrera wrote on 8/3/20 4:54 PM:On 2020-Aug-03, Ben Chobot wrote:
Alvaro Herrera wrote on 8/3/20 2:34 PM:
On 2020-Aug-03, Ben Chobot wrote:
dd if=16605/16613/60529051 bs=8192 count=1 seek=6501 of=/tmp/page.6501
If I use skip instead of seek....
Argh, yes, I did correct that in my test and forgot to copy and paste.
lsn | checksum | flags | lower | upper | special | pagesize |
version | prune_xid
--------------+----------+-------+-------+-------+---------+----------+---------+-----------
A0A/99BA11F8 | -215 | 0 | 180 | 7240 | 8176 | 8192
| 4 | 0
As I understand what we're looking at, this means the WAL stream was
assuming this page was last touched by A0A/AB2C43D0, but the page itself
thinks it was last touched by A0A/99BA11F8, which means at least one write
to the page is missing?
Yeah, that's exactly what we're seeing. Somehow an older page version
was resurrected. Of course, this should never happen.
So my theory has been proved. What now?
Just to close the loop on this, we haven't seen the issue since we've stopped expanding our filesystems by moving LVM extents between devices, so while I don't know exactly where the bug lies, I feel it's quite likely
not in Postgres.