Обсуждение: Problem with pg_upgrade 8.3 to 9.1.4 - clog missing?!
hi
I just upgraded test copy of database of our customer (~ 600GB of data).
upgrade went fine, no errors. but vacuumdb -azv ended with an error:
=> vacuumdb --all --analyze -p 6665
vacuumdb: vacuuming database "client_db"
vacuumdb: vacuuming database "pg_audit"
vacuumdb: vacuuming database "postgres"
vacuumdb: vacuuming of database "postgres" failed: ERROR: could not access status of transaction 860626316
DETAIL: Could not open file "pg_clog/0334": No such file or directory.
I know we had these kind of errors before, but I thought they all got fixed.
What can I do to debug it more?
8.3 was 8.3.16.
8.3.16 is from rpms, 9.1.4 is from source compilation.
both have integer datatimes, and both are for 64bit linux.
pg_clog contains:
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04A4
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04A5
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04A6
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04A7
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04A8
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04A9
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04AA
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04AB
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04AC
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04AD
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04AE
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04AF
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04B0
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04B1
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04B2
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04B3
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04B4
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04B5
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04B6
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04B7
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04B8
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04B9
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04BA
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04BB
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04BC
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04BD
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04BE
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04BF
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04C0
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04C1
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04C2
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04C3
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04C4
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04C5
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04C6
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04C7
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04C8
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04C9
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04CA
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04CB
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04CC
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04CD
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04CE
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04CF
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04D0
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04D1
-rw------- 1 postgres postgres 262144 Jun 6 21:22 04D2
-rw------- 1 postgres postgres 196608 Jun 7 00:15 04D3
unfortunately i do not have the 8.3 data dir any more.
if the problem is only with postgres database - i can ignore it. but I'd rather
be sure what is the source of the issue.
Best regards,
depesz
--
The best thing about modern society is how easy it is to avoid contact with it.
http://depesz.com/
On Thu, Jun 07, 2012 at 09:16:49PM +0200, hubert depesz lubaczewski wrote: > hi > I just upgraded test copy of database of our customer (~ 600GB of data). > upgrade went fine, no errors. but vacuumdb -azv ended with an error: > > => vacuumdb --all --analyze -p 6665 > vacuumdb: vacuuming database "client_db" > vacuumdb: vacuuming database "pg_audit" > vacuumdb: vacuuming database "postgres" > vacuumdb: vacuuming of database "postgres" failed: ERROR: could not access status of transaction 860626316 > DETAIL: Could not open file "pg_clog/0334": No such file or directory. > > I know we had these kind of errors before, but I thought they all got fixed. > > What can I do to debug it more? > > 8.3 was 8.3.16. > 8.3.16 is from rpms, 9.1.4 is from source compilation. > > both have integer datatimes, and both are for 64bit linux. I assume 8.3 vacuumdb did not generate these errors. If it was 8.4 I would suggest it was because we don't copy visibility map files from pre-9.1 because those were not crash-safe, but 8.3 didn't have visibility map files (added in PG 8.4), so 8.3 should have been checking the exact same transaction ids as 9.1. Basically, I can't think of a cause. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
On Wed, Jun 13, 2012 at 02:32:21PM -0400, Bruce Momjian wrote:
> I assume 8.3 vacuumdb did not generate these errors. If it was 8.4 I
> would suggest it was because we don't copy visibility map files from
> pre-9.1 because those were not crash-safe, but 8.3 didn't have
> visibility map files (added in PG 8.4), so 8.3 should have been checking
> the exact same transaction ids as 9.1.
>
> Basically, I can't think of a cause.
yeah, neither can we. but - any other test - even using the same source
of $PGDATA went flawless. I'm tempted to assume it's just a random
glitch in matrix, and just go on.
Best regards,
depesz
--
The best thing about modern society is how easy it is to avoid contact with it.
http://depesz.com/
On Wed, Jun 13, 2012 at 08:43:23PM +0200, hubert depesz lubaczewski wrote: > On Wed, Jun 13, 2012 at 02:32:21PM -0400, Bruce Momjian wrote: > > I assume 8.3 vacuumdb did not generate these errors. If it was 8.4 I > > would suggest it was because we don't copy visibility map files from > > pre-9.1 because those were not crash-safe, but 8.3 didn't have > > visibility map files (added in PG 8.4), so 8.3 should have been checking > > the exact same transaction ids as 9.1. > > > > Basically, I can't think of a cause. > > yeah, neither can we. but - any other test - even using the same source > of $PGDATA went flawless. I'm tempted to assume it's just a random > glitch in matrix, and just go on. Is the vacuumdb still repeatedly failing? -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
On Wed, Jun 13, 2012 at 02:55:41PM -0400, Bruce Momjian wrote: > On Wed, Jun 13, 2012 at 08:43:23PM +0200, hubert depesz lubaczewski wrote: > > On Wed, Jun 13, 2012 at 02:32:21PM -0400, Bruce Momjian wrote: > > > I assume 8.3 vacuumdb did not generate these errors. If it was 8.4 I > > > would suggest it was because we don't copy visibility map files from > > > pre-9.1 because those were not crash-safe, but 8.3 didn't have > > > visibility map files (added in PG 8.4), so 8.3 should have been checking > > > the exact same transaction ids as 9.1. > > > > > > Basically, I can't think of a cause. > > > > yeah, neither can we. but - any other test - even using the same source > > of $PGDATA went flawless. I'm tempted to assume it's just a random > > glitch in matrix, and just go on. > > Is the vacuumdb still repeatedly failing? well - in this particular upgraded database - i think so. I did run it couple of times, it failed all the times. But this $PGDATA is not longer used - we did another tests, and no other test showed problems - i.e. vacuum worked fine. Best regards, depesz