Обсуждение: pg_dumpall failure - could not access status of transaction

I am attempting to get a handle on the corruption that caused
the following error:

pg_dump: SQL command failed
pg_dump: Error message from server: ERROR:  could not access status of
transaction 3183024
DETAIL:  could not open file "pg_clog/0003": No such file or directory
pg_dump: The command was: SELECT description, classoid, objoid, objsubid
FROM pg_catalog.pg_description ORDER BY classoid, objoid, objsubid

I've run pg_filedump on all the files with the name = oid from:

select oid,relname from pg_class where relname='pg_description';
 oid  |          relname
------+----------------------------
 2609 | pg_description

but there are no errors reported.

I'm (obviously) lost here - what is the correct way to track down the
appropriate file and record to point pg_filedump at to view any
corruption?

Robert

I wrote:
> I am attempting to get a handle on the corruption that caused
> the following error...

After a bit more digging I found a couple of old messages that helped -
http://archives.postgresql.org/pgsql-admin/2004-01/msg00258.php

The last tuple that is valid is (3, 123) and looking at record 3 in the
corresponding file, it appears to be the last entry in that record and
it has no errors. Assuming that the next tuple will be in record 4 and
dumping it, results in the first 32 items having the error "Error:
Computed header length not equal to header size. Computed <28>  Header:
<0>" or similar.

Is there any likelihood this is a pg problem (it is running 8.1.4 on
ubuntu 6.06 server)? Is upgrading to 8.2.x worthwhile or is it more
likely to be a h/w issue? Note that the same box had 'invalid page
header' issues around 4 weeks ago - although these have not occurred
since the supplier replaced the shared memory video card with a
non-shared card 2 weeks ago. Basically, it would be good to be able to
go to the supplier and say there is  definitely a h/w issue here, so any
help in this area would be great.

Thanks

Robert

Robert Leftwich wrote:
> pg_dump: Error message from server: ERROR:  could not access status
> of transaction 3183024
> DETAIL:  could not open file "pg_clog/0003": No such file or
> directory

If your database system has already experienced on the order of 3183024
transactions, then you should wonder where that file went.  Otherwise
you probably have garbage on disk, pointing to a transaction that
hasn't happened yet.  This is usually hardware trouble in any case,
unless you have actually deleted the file manually.

The alternative is that your database system has already seen more than
2^32 transactions and you have hit some bug in the transaction
wraparound, but that would be much less likely than the above cases.

> I'm (obviously) lost here - what is the correct way to track down the
> appropriate file and record to point pg_filedump at to view any
> corruption?

Look for something that contains 3183024 as a four-byte integer.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut <peter_e@gmx.net> writes:
> Robert Leftwich wrote:
>> pg_dump: Error message from server: ERROR:  could not access status
>> of transaction 3183024
>> DETAIL:  could not open file "pg_clog/0003": No such file or
>> directory

> If your database system has already experienced on the order of 3183024
> transactions, then you should wonder where that file went.

Yeah.  What file names are actually present in $PGDATA/pg_clog/, and
what does pg_controldata say about the current XID counter?  We have
seen some bugs where clog files get deleted before they're not needed
anymore :-(

Also, can you show us the pg_filedump output for the page of
pg_description you think is trashed?  I find "-i -f" options give
the most useful output usually.

            regards, tom lane

On Fri, 02 Feb 2007 21:21:47 -0500, "Tom Lane" <tgl@sss.pgh.pa.us> said:
>
> Yeah.  What file names are actually present in $PGDATA/pg_clog/, and
> what does pg_controldata say about the current XID counter?  We have
> seen some bugs where clog files get deleted before they're not needed
> anymore :-(

pg_clog only has 2 files:

0000  0001

pg_controldata's only reference to XID is:

Latest checkpoint's NextXID:          1927588

>
> Also, can you show us the pg_filedump output for the page of
> pg_description you think is trashed?  I find "-i -f" options give
> the most useful output usually.
>

Block    4 ********************************************************
<Header> -----
 Block Offset: 0x00008000         Offsets: Lower     536 (0x0218)
 Block: Size 8192  Version    3            Upper     556 (0x022c)
 LSN:  logid      0 recoff 0x0018440c      Special  8192 (0x2000)
 Items:  129                   Free Space:   20
 Length (including item array): 540

  0000: 00000000 0c441800 01000000 18022c02  .....D........,.
  0010: 00200320 cc9f6800 989f6400 649f6800  . . ..h...d.d.h.
  0020: 309f6400 009f5e00 d09e5e00 9c9e6200  0.d...^...^...b.
  0030: 649e6a00 2c9e6a00 ec9d7c00 b49d7000  d.j.,.j...|...p.
  0040: 709d8200 409d5e00 0c9d6800 d89c6800  p...@.^...h...h.
  0050: a49c6400 689c7200 2c9c7400 f89b6800  ..d.h.r.,.t...h.
  0060: b49b8400 809b6600 4c9b6600 189b6600  ......f.L.f...f.
  0070: e49a6600 b09a6600 7c9a6600 3c9a8000  ..f...f.|.f.<...
  0080: fc998000 ac999e00 5c999e00 1c997a00  ........\.....z.
  0090: dc987c00 9c987a00 5c987c00 1c987a00  ..|...z.\.|...z.
  00a0: dc977e00 9c977e00 58978200 1c977400  ..~...~.X.....t.
  00b0: e0967200 9c968200 4896a200 00969000  ..r.....H.......
  00c0: b8958e00 80956c00 4c956800 14956a00  ......l.L.h...j.
  00d0: dc946c00 98948400 54948200 08949400  ..l.....T.......
  00e0: d0937000 98936a00 60937000 2c936200  ..p...j.`.p.,.b.
  00f0: ec927c00 a8928200 60928e00 28926a00  ..|.....`...(.j.
  0100: e8917e00 a4918200 68917800 28918000  ..~.....h.x.(...
  0110: dc909400 a4906c00 6c906a00 30907400  ......l.l.j.0.t.
  0120: f88f6a00 c08f6e00 8c8f6200 4c8f8000  ..j...n...b.L...
  0130: 0c8f8000 cc8e8000 8c8e8000 488e8400  ............H...
  0140: 048e8400 b48d9a00 7c8d6c00 448d6c00  ........|.l.D.l.
  0150: 148d5e00 e48c5e00 a48c7e00 688c7600  ..^...^...~.h.v.
  0160: 248c8400 d88b9600 9c8b7200 6c8b5e00  $.........r.l.^.
  0170: 3c8b5e00 0c8b5e00 dc8a5e00 a88a6200  <.^...^...^...b.
  0180: 708a6a00 308a7c00 f8897000 b4898200  p.j.0.|...p.....
  0190: 7c896a00 40897200 04897400 c4887c00  |.j.@.r...t...|.
  01a0: 94886000 3888b800 08885e00 d8875e00  ..`.8.....^...^.
  01b0: a4876200 6c876a00 2c877c00 f4867000  ..b.l.j.,.|...p.
  01c0: b0868200 78866a00 38867c00 00866a00  ....x.j.8.|...j.
  01d0: c0857c00 88857000 44858200 0c856a00  ..|...p.D.....j.
  01e0: cc847c00 90847200 54847400 20846800  ..|...r.T.t. .h.
  01f0: f0835e00 bc836800 8c835e00 5c835e00  ..^...h...^.\.^.
  0200: 28836200 f8825e00 c4826800 90826800  (.b...^...h...h.
  0210: 5c826400 2c825e00 00000000           \.d.,.^.....

<Data> ------
 Item   1 -- Length:   52  Offset: 8140 (0x1fcc)  Flags: USED
  XMIN: 0  CMIN: 0  XMAX: 0  CMAX|XVAC: 0
  Block Id: 0  linp Index: 0   Attributes: 0   Size: 0
  infomask: 0x0200 (XMIN_INVALID)
  Error: Computed header length not equal to header size.
         Computed <28>  Header: <0>

  1fcc: 00000000 00000000 00000000 00000000  ................
  1fdc: 00000000 00000000 00020000 00000000  ................
  1fec: 00000000 00000000 00000000 00000000  ................
  1ffc: 00000000                             ....

 Item   2 -- Length:   50  Offset: 8088 (0x1f98)  Flags: USED
  XMIN: 0  CMIN: 0  XMAX: 0  CMAX|XVAC: 0
  Block Id: 0  linp Index: 0   Attributes: 0   Size: 0
  infomask: 0x0200 (XMIN_INVALID)
  Error: Computed header length not equal to header size.
         Computed <28>  Header: <0>

  1f98: 00000000 00000000 00000000 00000000  ................
  1fa8: 00000000 00000000 00020000 00000000  ................
  1fb8: 00000000 00000000 00000000 00000000  ................
  1fc8: 0000                                 ..

 Item   3 -- Length:   52  Offset: 8036 (0x1f64)  Flags: USED
  XMIN: 0  CMIN: 0  XMAX: 0  CMAX|XVAC: 0
  Block Id: 0  linp Index: 0   Attributes: 0   Size: 0
  infomask: 0x0200 (XMIN_INVALID)
  Error: Computed header length not equal to header size.
         Computed <28>  Header: <0>

  1f64: 00000000 00000000 00000000 00000000  ................
  1f74: 00000000 00000000 00020000 00000000  ................
  1f84: 00000000 00000000 00000000 00000000  ................
  1f94: 00000000                             ....

 Item   4 -- Length:   50  Offset: 7984 (0x1f30)  Flags: USED
  XMIN: 0  CMIN: 0  XMAX: 0  CMAX|XVAC: 0
  Block Id: 0  linp Index: 0   Attributes: 0   Size: 0
  infomask: 0x0200 (XMIN_INVALID)
  Error: Computed header length not equal to header size.
         Computed <28>  Header: <0>

  1f30: 00000000 00000000 00000000 00000000  ................
  1f40: 00000000 00000000 00020000 00000000  ................
  1f50: 00000000 00000000 00000000 00000000  ................
  1f60: 0000                                 ..

 Item   5 -- Length:   47  Offset: 7936 (0x1f00)  Flags: USED
  XMIN: 0  CMIN: 0  XMAX: 0  CMAX|XVAC: 0
  Block Id: 0  linp Index: 0   Attributes: 0   Size: 0
  infomask: 0x0200 (XMIN_INVALID)
  Error: Computed header length not equal to header size.
         Computed <28>  Header: <0>

  1f00: 00000000 00000000 00000000 00000000  ................
  1f10: 00000000 00000000 00020000 00000000  ................
  1f20: 00000000 00000000 00000000 000000    ...............

 Item   6 -- Length:   47  Offset: 7888 (0x1ed0)  Flags: USED
  XMIN: 0  CMIN: 0  XMAX: 0  CMAX|XVAC: 0
  Block Id: 0  linp Index: 0   Attributes: 0   Size: 0
  infomask: 0x0200 (XMIN_INVALID)
  Error: Computed header length not equal to header size.
         Computed <28>  Header: <0>

  1ed0: 00000000 00000000 00000000 00000000  ................
  1ee0: 00000000 00000000 00020000 00000000  ................
  1ef0: 00000000 00000000 00000000 000000    ...............

 Item   7 -- Length:   49  Offset: 7836 (0x1e9c)  Flags: USED
  XMIN: 0  CMIN: 0  XMAX: 0  CMAX|XVAC: 0
  Block Id: 0  linp Index: 0   Attributes: 0   Size: 0
  infomask: 0x0200 (XMIN_INVALID)
  Error: Computed header length not equal to header size.
         Computed <28>  Header: <0>

  1e9c: 00000000 00000000 00000000 00000000  ................
  1eac: 00000000 00000000 00020000 00000000  ................
  1ebc: 00000000 00000000 00000000 00000000  ................
  1ecc: 00                                   .

 Item   8 -- Length:   53  Offset: 7780 (0x1e64)  Flags: USED
  XMIN: 0  CMIN: 0  XMAX: 0  CMAX|XVAC: 0
  Block Id: 0  linp Index: 0   Attributes: 0   Size: 0
  infomask: 0x0200 (XMIN_INVALID)
  Error: Computed header length not equal to header size.
         Computed <28>  Header: <0>

  1e64: 00000000 00000000 00000000 00000000  ................
  1e74: 00000000 00000000 00020000 00000000  ................
  1e84: 00000000 00000000 00000000 00000000  ................
  1e94: 00000000 00                          .....

 Item   9 -- Length:   53  Offset: 7724 (0x1e2c)  Flags: USED
  XMIN: 0  CMIN: 0  XMAX: 0  CMAX|XVAC: 0
  Block Id: 0  linp Index: 0   Attributes: 0   Size: 0
  infomask: 0x0200 (XMIN_INVALID)
  Error: Computed header length not equal to header size.
         Computed <28>  Header: <0>

  1e2c: 00000000 00000000 00000000 00000000  ................
  1e3c: 00000000 00000000 00020000 00000000  ................
  1e4c: 00000000 00000000 00000000 00000000  ................
  1e5c: 00000000 00                          .....

 Item  10 -- Length:   62  Offset: 7660 (0x1dec)  Flags: USED
  XMIN: 0  CMIN: 0  XMAX: 0  CMAX|XVAC: 0
  Block Id: 0  linp Index: 0   Attributes: 0   Size: 0
  infomask: 0x0200 (XMIN_INVALID)
  Error: Computed header length not equal to header size.
         Computed <28>  Header: <0>

  1dec: 00000000 00000000 00000000 00000000  ................
  1dfc: 00000000 00000000 00020000 00000000  ................
  1e0c: 00000000 00000000 00000000 00000000  ................
  1e1c: 00000000 00000000 00000000 0000      ..............

 Item  11 -- Length:   56  Offset: 7604 (0x1db4)  Flags: USED
  XMIN: 0  CMIN: 0  XMAX: 0  CMAX|XVAC: 0
  Block Id: 0  linp Index: 0   Attributes: 0   Size: 0
  infomask: 0x0200 (XMIN_INVALID)
  Error: Computed header length not equal to header size.
         Computed <28>  Header: <0>

  1db4: 00000000 00000000 00000000 00000000  ................
  1dc4: 00000000 00000000 00020000 00000000  ................
  1dd4: 00000000 00000000 00000000 00000000  ................
  1de4: 00000000 00000000                    ........

 Item  12 -- Length:   65  Offset: 7536 (0x1d70)  Flags: USED
  XMIN: 0  CMIN: 0  XMAX: 0  CMAX|XVAC: 0
  Block Id: 0  linp Index: 0   Attributes: 0   Size: 0
  infomask: 0x0200 (XMIN_INVALID)
  Error: Computed header length not equal to header size.
         Computed <28>  Header: <0>

  1d70: 00000000 00000000 00000000 00000000  ................
  1d80: 00000000 00000000 00020000 00000000  ................
  1d90: 00000000 00000000 00000000 00000000  ................
  1da0: 00000000 00000000 00000000 00000000  ................
  1db0: 00                                   .

 Item  13 -- Length:   47  Offset: 7488 (0x1d40)  Flags: USED
  XMIN: 0  CMIN: 0  XMAX: 0  CMAX|XVAC: 0
  Block Id: 0  linp Index: 0   Attributes: 0   Size: 0
  infomask: 0x0200 (XMIN_INVALID)
  Error: Computed header length not equal to header size.
         Computed <28>  Header: <0>

  1d40: 00000000 00000000 00000000 00000000  ................
  1d50: 00000000 00000000 00020000 00000000  ................
  1d60: 00000000 00000000 00000000 000000    ...............

 Item  14 -- Length:   52  Offset: 7436 (0x1d0c)  Flags: USED
  XMIN: 0  CMIN: 0  XMAX: 0  CMAX|XVAC: 0
  Block Id: 0  linp Index: 0   Attributes: 0   Size: 0
  infomask: 0x0200 (XMIN_INVALID)
  Error: Computed header length not equal to header size.
         Computed <28>  Header: <0>

  1d0c: 00000000 00000000 00000000 00000000  ................
  1d1c: 00000000 00000000 00020000 00000000  ................
  1d2c: 00000000 00000000 00000000 00000000  ................
  1d3c: 00000000                             ....

 Item  15 -- Length:   52  Offset: 7384 (0x1cd8)  Flags: USED
  XMIN: 0  CMIN: 0  XMAX: 0  CMAX|XVAC: 0
  Block Id: 0  linp Index: 0   Attributes: 0   Size: 0
  infomask: 0x0200 (XMIN_INVALID)
  Error: Computed header length not equal to header size.
         Computed <28>  Header: <0>

  1cd8: 00000000 00000000 00000000 00000000  ................
  1ce8: 00000000 00000000 00020000 00000000  ................
  1cf8: 00000000 00000000 00000000 00000000  ................
  1d08: 00000000                             ....

 Item  16 -- Length:   50  Offset: 7332 (0x1ca4)  Flags: USED
  XMIN: 0  CMIN: 0  XMAX: 0  CMAX|XVAC: 0
  Block Id: 0  linp Index: 0   Attributes: 0   Size: 0
  infomask: 0x0200 (XMIN_INVALID)
  Error: Computed header length not equal to header size.
         Computed <28>  Header: <0>

  1ca4: 00000000 00000000 00000000 00000000  ................
  1cb4: 00000000 00000000 00020000 00000000  ................
  1cc4: 00000000 00000000 00000000 00000000  ................
  1cd4: 0000                                 ..

 Item  17 -- Length:   57  Offset: 7272 (0x1c68)  Flags: USED
  XMIN: 3183024  CMIN: 3183000  XMAX: 3182976  CMAX|XVAC: 3182952
  Block Id: 0  linp Index: 0   Attributes: 0   Size: 0
  infomask: 0x0000 ()
  Error: Computed header length not equal to header size.
         Computed <28>  Header: <0>

  1c68: b0913000 98913000 80913000 68913000  ..0...0...0.h.0.
  1c78: 00000000 00000000 00000000 00000000  ................
  1c88: 00000000 00000000 00000000 00000000  ................
  1c98: 00000000 00000000 00                 .........

 Item  18 -- Length:   58  Offset: 7212 (0x1c2c)  Flags: USED
  XMIN: 3183384  CMIN: 3183360  XMAX: 3183336  CMAX|XVAC: 3183312
  Block Id: 2461532208  linp Index: 37536   Attributes: 48   Size: 48
  infomask: 0x9288
  (HASCOMPRESSED|XMAX_SHARED_LOCK|XMIN_INVALID|XMAX_IS_MULTI|MOVED_IN)
  Error: Computed header length not equal to header size.
         Computed <28>  Header: <48>

  1c2c: 18933000 00933000 e8923000 d0923000  ..0...0...0...0.
  1c3c: b8923000 a0923000 88923000 70923000  ..0...0...0.p.0.
  1c4c: 58923000 40923000 28923000 10923000  X.0.@.0.(.0...0.
  1c5c: f8913000 e0913000 c891               ..0...0...

 Item  19 -- Length:   52  Offset: 7160 (0x1bf8)  Flags: USED
  XMIN: 3183696  CMIN: 3183672  XMAX: 3183648  CMAX|XVAC: 3183624
  Block Id: 2481979440  linp Index: 37848   Attributes: 48   Size: 48
  infomask: 0x93c0
  (XMAX_EXCL_LOCK|XMAX_SHARED_LOCK|XMIN_COMMITTED|XMIN_INVALID|XMAX_IS_MULTI|MOVED_IN)
  Error: Computed header length not equal to header size.
         Computed <28>  Header: <48>

  1bf8: 50943000 38943000 20943000 08943000  P.0.8.0. .0...0.
  1c08: f0933000 d8933000 c0933000 a8933000  ..0...0...0...0.
  1c18: 90933000 78933000 60933000 48933000  ..0.x.0.`.0.H.0.
  1c28: 30933000                             0.0.

 Item  20 -- Length:   66  Offset: 7092 (0x1bb4)  Flags: USED
  XMIN: 3184104  CMIN: 3184080  XMAX: 3184056  CMAX|XVAC: 3184032  OID:
  3183840
  Block Id: 2508718128  linp Index: 38256   Attributes: 48   Size: 48
  infomask: 0x9558
  (HASCOMPRESSED|HASOID|XMAX_EXCL_LOCK|XMIN_COMMITTED|XMAX_COMMITTED|XMAX_IS_MULTI|MOVED_IN)
  Error: Computed header length not equal to header size.
         Computed <32>  Header: <48>

  1bb4: e8953000 d0953000 b8953000 a0953000  ..0...0...0...0.
  1bc4: 88953000 70953000 58953000 40953000  ..0.p.0.X.0.@.0.
  1bd4: 28953000 10953000 f8943000 e0943000  (.0...0...0...0.
  1be4: c8943000 b0943000 98943000 80943000  ..0...0...0...0.
  1bf4: 6894                                 h.

 Item  21 -- Length:   51  Offset: 7040 (0x1b80)  Flags: USED
  XMIN: 3184416  CMIN: 3184392  XMAX: 3184368  CMAX|XVAC: 3184344  OID:
  3184152
  Block Id: 2529165360  linp Index: 38568   Attributes: 48   Size: 48
  infomask: 0x9690
  (HASOID|XMAX_SHARED_LOCK|XMIN_INVALID|XMAX_COMMITTED|XMAX_IS_MULTI|MOVED_IN)
  Error: Computed header length not equal to header size.
         Computed <32>  Header: <48>

  1b80: 20973000 08973000 f0963000 d8963000   .0...0...0...0.
  1b90: c0963000 a8963000 90963000 78963000  ..0...0...0.x.0.
  1ba0: 60963000 48963000 30963000 18963000  `.0.H.0.0.0...0.
  1bb0: 009630                               ..0

 Item  22 -- Length:   51  Offset: 6988 (0x1b4c)  Flags: USED
  XMIN: 3184728  CMIN: 3184704  XMAX: 3184680  CMAX|XVAC: 3184656
  Block Id: 2549612592  linp Index: 38880   Attributes: 48   Size: 48
  infomask: 0x97c8
  (HASCOMPRESSED|XMAX_EXCL_LOCK|XMAX_SHARED_LOCK|XMIN_COMMITTED|XMIN_INVALID|XMAX_COMMITTED|XMAX_IS_MULTI|MOVED_IN)
  Error: Computed header length not equal to header size.
         Computed <28>  Header: <48>

  1b4c: 58983000 40983000 28983000 10983000  X.0.@.0.(.0...0.
  1b5c: f8973000 e0973000 c8973000 b0973000  ..0...0...0...0.
  1b6c: 98973000 80973000 68973000 50973000  ..0...0.h.0.P.0.
  1b7c: 389730                               8.0

 Item  23 -- Length:   51  Offset: 6936 (0x1b18)  Flags: USED
  XMIN: 3247672  CMIN: 3247720  XMAX: 3247768  CMAX|XVAC: 3247816  OID:
  3247240
  Block Id: 2398617649  linp Index: 36648   Attributes: 49   Size: 48
  infomask: 0x8b98
  (HASCOMPRESSED|HASOID|XMAX_SHARED_LOCK|XMIN_COMMITTED|XMIN_INVALID|XMAX_INVALID|MOVED_IN)
  Error: Computed header length not equal to header size.
         Computed <32>  Header: <48>

  1b18: 388e3100 688e3100 988e3100 c88e3100  8.1.h.1...1...1.
  1b28: f88e3100 288f3100 988b3000 c88b3100  ..1.(.1...0...1.
  1b38: f88b3100 288c3100 588c3100 888c3100  ..1.(.1.X.1...1.
  1b48: 709830                               p.0

 Item  24 -- Length:   51  Offset: 6884 (0x1ae4)  Flags: USED
  XMIN: 3248056  CMIN: 3248104  XMAX: 3248152  CMAX|XVAC: 3248200
  Block Id: 2423783473  linp Index: 36024   Attributes: 48   Size: 49
  infomask: 0x8ce8
  (HASCOMPRESSED|XMAX_EXCL_LOCK|XMAX_SHARED_LOCK|XMAX_COMMITTED|XMAX_INVALID|MOVED_IN)
  Error: Computed header length not equal to header size.
         Computed <28>  Header: <49>

  1ae4: b88f3100 e88f3100 18903100 48903100  ..1...1...1.H.1.
  1af4: 78903100 b88c3000 e88c3100 188d3100  x.1...0...1...1.
  1b04: 488d3100 788d3100 a88d3100 d88d3100  H.1.x.1...1...1.
  1b14: 088e31                               ..1

 Item  25 -- Length:   51  Offset: 6832 (0x1ab0)  Flags: USED
  XMIN: 3184920  CMIN: 3184896  XMAX: 3184872  CMAX|XVAC: 3184848
  Block Id: 2562195504  linp Index: 39072   Attributes: 48   Size: 48
  infomask: 0x90a8
  (HASCOMPRESSED|XMAX_SHARED_LOCK|XMAX_IS_MULTI|MOVED_IN)
  Error: Computed header length not equal to header size.
         Computed <28>  Header: <48>

  1ab0: 18993000 00993000 e8983000 d0983000  ..0...0...0...0.
  1ac0: b8983000 a0983000 a8903000 d8903100  ..0...0...0...1.
  1ad0: 08913100 38913100 88983100 588f3000  ..1.8.1...1.X.0.
  1ae0: 888f31                               ..1

 Item  26 -- Length:   51  Offset: 6780 (0x1a7c)  Flags: USED
  XMIN: 3185232  CMIN: 3185208  XMAX: 3185184  CMAX|XVAC: 3185160
  Block Id: 2582642736  linp Index: 39384   Attributes: 48   Size: 48
  infomask: 0x99c0
  (XMAX_EXCL_LOCK|XMAX_SHARED_LOCK|XMIN_COMMITTED|XMAX_INVALID|XMAX_IS_MULTI|MOVED_IN)
  Error: Computed header length not equal to header size.
         Computed <28>  Header: <48>

  1a7c: 509a3000 389a3000 209a3000 089a3000  P.0.8.0. .0...0.
  1a8c: f0993000 d8993000 c0993000 a8993000  ..0...0...0...0.
  1a9c: 90993000 78993000 60993000 48993000  ..0.x.0.`.0.H.0.
  1aac: 309930                               0.0

 Item  27 -- Length:   64  Offset: 6716 (0x1a3c)  Flags: USED
  XMIN: 3185616  CMIN: 3185592  XMAX: 3185568  CMAX|XVAC: 3185544
  Block Id: 2607808560  linp Index: 39768   Attributes: 48   Size: 48
  infomask: 0x9b40
  (XMAX_EXCL_LOCK|XMIN_COMMITTED|XMIN_INVALID|XMAX_INVALID|XMAX_IS_MULTI|MOVED_IN)
  Error: Computed header length not equal to header size.
         Computed <28>  Header: <48>

  1a3c: d09b3000 b89b3000 a09b3000 889b3000  ..0...0...0...0.
  1a4c: 709b3000 589b3000 409b3000 289b3000  p.0.X.0.@.0.(.0.
  1a5c: 109b3000 f89a3000 e09a3000 c89a3000  ..0...0...0...0.
  1a6c: b09a3000 989a3000 809a3000 689a3000  ..0...0...0.h.0.

 Item  28 -- Length:   64  Offset: 6652 (0x19fc)  Flags: USED
  XMIN: 3247168  CMIN: 3247216  XMAX: 3247264  CMAX|XVAC: 3185928
  Block Id: 2632974384  linp Index: 40152   Attributes: 48   Size: 48
  infomask: 0x9cc0
  (XMAX_EXCL_LOCK|XMAX_SHARED_LOCK|XMAX_COMMITTED|XMAX_INVALID|XMAX_IS_MULTI|MOVED_IN)
  Error: Computed header length not equal to header size.
         Computed <28>  Header: <48>

  19fc: 408c3100 708c3100 a08c3100 089d3000  @.1.p.1...1...0.
  1a0c: f09c3000 d89c3000 c09c3000 a89c3000  ..0...0...0...0.
  1a1c: 909c3000 789c3000 609c3000 489c3000  ..0.x.0.`.0.H.0.
  1a2c: 309c3000 189c3000 009c3000 e89b3000  0.0...0...0...0.

 Item  29 -- Length:   79  Offset: 6572 (0x19ac)  Flags: USED
  XMIN: 3248176  CMIN: 3248224  XMAX: 3248272  CMAX|XVAC: 3181776
  Block Id: 2365587505  linp Index: 36144   Attributes: 49   Size: 49
  infomask: 0x8d60
  (XMAX_EXCL_LOCK|XMIN_COMMITTED|XMAX_COMMITTED|XMAX_INVALID|MOVED_IN)
  Error: Computed header length not equal to header size.
         Computed <28>  Header: <49>

  19ac: 30903100 60903100 90903100 d08c3000  0.1.`.1...1...0.
  19bc: 008d3100 308d3100 608d3100 908d3100  ..1.0.1.`.1...1.
  19cc: c08d3100 f08d3100 208e3100 508e3100  ..1...1. .1.P.1.
  19dc: 808e3100 b08e3100 e08e3100 108f3100  ..1...1...1...1.
  19ec: 408f3100 b08b3000 e08b3100 108c31    @.1...0...1...1

 Item  30 -- Length:   79  Offset: 6492 (0x195c)  Flags: USED
  XMIN: 3186216  CMIN: 3186192  XMAX: 3186168  CMAX|XVAC: 3186144  OID:
  3182784
  Block Id: 2647130160  linp Index: 40368   Attributes: 48   Size: 48
  infomask: 0x9d98
  (HASCOMPRESSED|HASOID|XMAX_SHARED_LOCK|XMIN_COMMITTED|XMAX_COMMITTED|XMAX_INVALID|XMAX_IS_MULTI|MOVED_IN)
  Error: Computed header length not equal to header size.
         Computed <32>  Header: <48>

  195c: 289e3000 109e3000 f89d3000 e09d3000  (.0...0...0...0.
  196c: c89d3000 b09d3000 989d3000 809d3000  ..0...0...0...0.
  197c: 689d3000 509d3000 389d3000 c0903000  h.0.P.0.8.0...0.
  198c: f0903100 20913100 50913100 209d3100  ..1. .1.P.1. .1.
  199c: 708f3000 a08f3100 d08f3100 009031    p.0...1...1...1

 Item  31 -- Length:   61  Offset: 6428 (0x191c)  Flags: USED
  XMIN: 3186600  CMIN: 3186576  XMAX: 3186552  CMAX|XVAC: 3186528  OID:
  3186336
  Block Id: 2672295984  linp Index: 40752   Attributes: 48   Size: 48
  infomask: 0x9f18
  (HASCOMPRESSED|HASOID|XMIN_COMMITTED|XMIN_INVALID|XMAX_COMMITTED|XMAX_INVALID|XMAX_IS_MULTI|MOVED_IN)
  Error: Computed header length not equal to header size.
         Computed <32>  Header: <48>

  191c: a89f3000 909f3000 789f3000 609f3000  ..0...0.x.0.`.0.
  192c: 489f3000 309f3000 189f3000 009f3000  H.0.0.0...0...0.
  193c: e89e3000 d09e3000 b89e3000 a09e3000  ..0...0...0...0.
  194c: 889e3000 709e3000 589e3000 40        ..0.p.0.X.0.@

 Item  32 -- Length:   62  Offset: 6364 (0x18dc)  Flags: USED
  XMIN: 2  CMIN: 0  XMAX: 0  CMAX|XVAC: 0
  Block Id: 4  linp Index: 32   Attributes: 4   Size: 28
  infomask: 0x0902 (HASVARWIDTH|XMIN_COMMITTED|XMAX_INVALID)

  18dc: 02000000 00000000 00000000 00000000  ................
  18ec: 00000400 20000400 02091c00 b9030000  .... ...........
  18fc: e7040000 14000000 78dfec73 01000000  ........x..s....
  190c: 7803980b f01f0320 d89f3000 c09f      x...... ..0...

SNIP NO MORE ERRORS...

*** End of Requested Range Encountered. Last Block Read: 4 ***

"Robert Leftwich" <pg@rtl.fmailbox.com> writes:
> On Fri, 02 Feb 2007 21:21:47 -0500, "Tom Lane" <tgl@sss.pgh.pa.us> said:
>> Also, can you show us the pg_filedump output for the page of
>> pg_description you think is trashed?  I find "-i -f" options give
>> the most useful output usually.

> [ dump snipped ]

Yeah, the last part of that page has been trashed --- mostly zeroes, but
there's a bit with an odd data pattern that doesn't make much sense to
me:

>   1a3c: d09b3000 b89b3000 a09b3000 889b3000  ..0...0...0...0.
>   1a4c: 709b3000 589b3000 409b3000 289b3000  p.0.X.0.@.0.(.0.
>   1a5c: 109b3000 f89a3000 e09a3000 c89a3000  ..0...0...0...0.
>   1a6c: b09a3000 989a3000 809a3000 689a3000  ..0...0...0.h.0.

This seems like it could indicate either a hardware or operating-system
flake-out.  Have you tried running hardware diagnostic tests on the
machine?

            regards, tom lane

On Fri, 02 Feb 2007 22:28:42 -0500, "Tom Lane" <tgl@sss.pgh.pa.us> said:
> This seems like it could indicate either a hardware or operating-system
> flake-out.  Have you tried running hardware diagnostic tests on the
> machine?
>

A couple of weeks back when it had 'invalid page header' problems,
memtest was run on it for 12-15hrs w/o a problem and smartmon didn't
report any errors from the drives - I'll check smartmon again.

Any other tests you would recommend?

Robert