Обсуждение: Upgrade Process Says "The database server was not shut downcleanly" but it was
I'm in the process to update my Postgres 10 to 12 (it's my first server upgrade). I've shutdown my Postgres 10 using "systemctl stop postgresql-10.service" and "systemctl status postgresql-10.service" says: Active: inactive (dead) since Sun 2020-05-03 22:51:46 CEST; After that I've started the upgrade process with: /usr/pgsql-12/bin/pg_upgrade --old-bindir=/usr/pgsql-10/bin/ --new-bindir=/usr/pgsql-12/bin/ --old-datadir=/var/lib/pgsql/10/data --new-datadir=/var/lib/pgsql/12/data --old-options '-c config_file=/var/lib/pgsql/10/data/postgresql.conf' --new-options '-c config_file=/var/lib/pgsql/12/data/postgresql.conf' And the output is: Performing Consistency Checks ----------------------------- Checking cluster versions ok Checking database user is the install user ok Checking database connection settings ok Checking for prepared transactions ok Checking for reg* data types in user tables ok Checking for contrib/isn with bigint-passing mismatch ok Checking for tables WITH OIDS ok Checking for invalid "sql_identifier" user columns ok Creating dump of global objects ok Creating dump of database schemas ok Checking for presence of required libraries ok Checking database user is the install user ok Checking for prepared transactions ok If pg_upgrade fails after this point, you must re-initdb the new cluster before continuing. Performing Upgrade ------------------ Analyzing all rows in the new cluster ok Freezing all rows in the new cluster ok Deleting files from new pg_xact ok Copying old pg_xact to new server ok Setting next transaction ID and epoch for new cluster ok Deleting files from new pg_multixact/offsets ok Copying old pg_multixact/offsets to new server ok Deleting files from new pg_multixact/members ok Copying old pg_multixact/members to new server ok Setting next multixact ID and offset for new cluster ok Resetting WAL archives ok Setting frozenxid and minmxid counters in new cluster ok Restoring global objects in the new cluster ok Restoring database schemas in the new cluster ok Copying user relation files ok Setting next OID for new cluster *failure* Consult the last few lines of "pg_upgrade_utility.log" for the probable cause of the failure. Failure, exiting The pg_upgrade_utility.log shows: command: "/usr/pgsql-12/bin/pg_resetwal" -o 1091293 "/var/lib/pgsql/12/data" >> "pg_upgrade_utility.log" 2>&1 The database server was not shut down cleanly. Resetting the write-ahead log might cause data to be lost. If you want to proceed anyway, use -f to force reset 1. I did shutdown the server properly "systemctl stop postgresql-10.service", is there a better way to do that? 2. Where do I set the "-f" flag if I choose to force reset? -- Sent from: https://www.postgresql-archive.org/PostgreSQL-general-f1843780.html
Re: Upgrade Process Says "The database server was not shut downcleanly" but it was
От
Adrian Klaver
Дата:
On 5/3/20 2:53 PM, TalGloz wrote: > I'm in the process to update my Postgres 10 to 12 (it's my first server > upgrade). I've shutdown my Postgres 10 using "systemctl stop > postgresql-10.service" and "systemctl status postgresql-10.service" says: > Active: inactive (dead) since Sun 2020-05-03 22:51:46 CEST; > > After that I've started the upgrade process with: > /usr/pgsql-12/bin/pg_upgrade --old-bindir=/usr/pgsql-10/bin/ > --new-bindir=/usr/pgsql-12/bin/ --old-datadir=/var/lib/pgsql/10/data > --new-datadir=/var/lib/pgsql/12/data --old-options '-c > config_file=/var/lib/pgsql/10/data/postgresql.conf' --new-options '-c > config_file=/var/lib/pgsql/12/data/postgresql.conf' > > And the output is: > Performing Consistency Checks > ----------------------------- > Checking cluster versions ok > Checking database user is the install user ok > Checking database connection settings ok > Checking for prepared transactions ok > Checking for reg* data types in user tables ok > Checking for contrib/isn with bigint-passing mismatch ok > Checking for tables WITH OIDS ok > Checking for invalid "sql_identifier" user columns ok > Creating dump of global objects ok > Creating dump of database schemas > > ok > Checking for presence of required libraries ok > Checking database user is the install user ok > Checking for prepared transactions ok > > If pg_upgrade fails after this point, you must re-initdb the > new cluster before continuing. > > Performing Upgrade > ------------------ > Analyzing all rows in the new cluster ok > Freezing all rows in the new cluster ok > Deleting files from new pg_xact ok > Copying old pg_xact to new server ok > Setting next transaction ID and epoch for new cluster ok > Deleting files from new pg_multixact/offsets ok > Copying old pg_multixact/offsets to new server ok > Deleting files from new pg_multixact/members ok > Copying old pg_multixact/members to new server ok > Setting next multixact ID and offset for new cluster ok > Resetting WAL archives ok > Setting frozenxid and minmxid counters in new cluster ok > Restoring global objects in the new cluster ok > Restoring database schemas in the new cluster > > ok > Copying user relation files > > ok > Setting next OID for new cluster > *failure* > > Consult the last few lines of "pg_upgrade_utility.log" for > the probable cause of the failure. > Failure, exiting > > The pg_upgrade_utility.log shows: > command: "/usr/pgsql-12/bin/pg_resetwal" -o 1091293 "/var/lib/pgsql/12/data" Looks like you did not shutdown the 12 instance. >>> "pg_upgrade_utility.log" 2>&1 > The database server was not shut down cleanly. > Resetting the write-ahead log might cause data to be lost. > If you want to proceed anyway, use -f to force reset > > 1. I did shutdown the server properly "systemctl stop > postgresql-10.service", is there a better way to do that? > 2. Where do I set the "-f" flag if I choose to force reset? > > > > -- > Sent from: https://www.postgresql-archive.org/PostgreSQL-general-f1843780.html > > -- Adrian Klaver adrian.klaver@aklaver.com
Adrian Klaver-4 wrote > On 5/3/20 2:53 PM, > > Looks like you did not shutdown the 12 instance. > > -- > Adrian Klaver > adrian.klaver@ This is weird because "systemctl staus postgresql-12.service" shows that the service is dead. Best regards, TalGloz -- Sent from: https://www.postgresql-archive.org/PostgreSQL-general-f1843780.html
Re: Upgrade Process Says "The database server was not shut downcleanly" but it was
От
Adrian Klaver
Дата:
On 5/3/20 3:11 PM, TalGloz wrote: > Adrian Klaver-4 wrote >> On 5/3/20 2:53 PM, >> >> Looks like you did not shutdown the 12 instance. >> >> -- >> Adrian Klaver > >> adrian.klaver@ > > > This is weird because "systemctl staus postgresql-12.service" shows that the > service is dead. It is now: "The database server was not shut down cleanly." Before running pg_upgrade again I would:\ 1) Start the 12 instance to make sure it will start cleanly. 2) Shut it and the 10 instance down. 3) Try the pg_upgrade again this time with: https://www.postgresql.org/docs/12/pgupgrade.html --check check clusters only, don't change any data 4) Assuming 3) completes, repeat w/o --check > > Best regards, > TalGloz > > > > -- > Sent from: https://www.postgresql-archive.org/PostgreSQL-general-f1843780.html > > -- Adrian Klaver adrian.klaver@aklaver.com
Adrian Klaver-4 wrote > On 5/3/20 3:11 PM, TalGloz wrote: >> Adrian Klaver-4 wrote >>> On 5/3/20 2:53 PM, >>> >>> Looks like you did not shutdown the 12 instance. >>> >>> -- >>> Adrian Klaver >> >>> adrian.klaver@ >> >> >> This is weird because "systemctl staus postgresql-12.service" shows that >> the >> service is dead. > > It is now: > "The database server was not shut down cleanly." > > Before running pg_upgrade again I would:\ > > 1) Start the 12 instance to make sure it will start cleanly. > > 2) Shut it and the 10 instance down. > > 3) Try the pg_upgrade again this time with: > > https://www.postgresql.org/docs/12/pgupgrade.html > > --check > > check clusters only, don't change any data > > 4) Assuming 3) completes, repeat w/o --check > >> >> Best regards, >> TalGloz >> >> >> >> -- >> Sent from: >> https://www.postgresql-archive.org/PostgreSQL-general-f1843780.html >> >> > > > -- > Adrian Klaver > adrian.klaver@ Thanks I'll try your steps tommorow. What I meant to say is that "systemctl staus postgresql-12.service" was always showing that the service is dead, even before I've started the upgrade process. Best regards, TalGloz -- Sent from: https://www.postgresql-archive.org/PostgreSQL-general-f1843780.html
Re: Upgrade Process Says "The database server was not shut downcleanly" but it was
От
Adrian Klaver
Дата:
On 5/3/20 3:33 PM, TalGloz wrote: > Adrian Klaver-4 wrote >> On 5/3/20 3:11 PM, TalGloz wrote: >>> Adrian Klaver-4 wrote >>>> On 5/3/20 2:53 PM, >>>> >>>> Looks like you did not shutdown the 12 instance. >>>> >>>> -- >>>> Adrian Klaver >>> >>>> adrian.klaver@ >>> >>> >>> This is weird because "systemctl staus postgresql-12.service" shows that >>> the >>> service is dead. >> >> It is now: >> "The database server was not shut down cleanly." >> >> Before running pg_upgrade again I would:\ >> >> 1) Start the 12 instance to make sure it will start cleanly. >> >> 2) Shut it and the 10 instance down. >> >> 3) Try the pg_upgrade again this time with: >> >> https://www.postgresql.org/docs/12/pgupgrade.html >> >> --check >> >> check clusters only, don't change any data >> >> 4) Assuming 3) completes, repeat w/o --check >> >>> >>> Best regards, >>> TalGloz >>> >>> >>> >>> -- >>> Sent from: >>> https://www.postgresql-archive.org/PostgreSQL-general-f1843780.html >>> >>> >> >> >> -- >> Adrian Klaver > >> adrian.klaver@ > > > Thanks I'll try your steps tommorow. What I meant to say is that "systemctl > staus postgresql-12.service" was always showing that the service is dead, > even before I've started the upgrade process. Hmm, then maybe the unclean shutdown was from a previous use. At at any rate you passed this point in the process: "If pg_upgrade fails after this point, you must re-initdb the new cluster before continuing." You are going to need to start with a fresh cluster anyway. > > Best regards, > TalGloz > > > > -- > Sent from: https://www.postgresql-archive.org/PostgreSQL-general-f1843780.html > > -- Adrian Klaver adrian.klaver@aklaver.com
Well I tried your steps, both Postgres 10 and 12 are in perfect running conditions and were shut down using the "systemctl" command. When I execute: /usr/pgsql-12/bin/pg_upgrade --old-bindir=/usr/pgsql-10/bin/ --new-bindir=/usr/pgsql-12/bin/ --old-datadir=/var/lib/pgsql/10/data --new-datadir=/var/lib/pgsql/12/data --old-options '-c config_file=/var/lib/pgsql/10/data/postgresql.conf' --new-options '-c config_file=/var/lib/pgsql/12/data/postgresql.conf' --check I get: Performing Consistency Checks ----------------------------- Checking cluster versions ok Checking database user is the install user ok Checking database connection settings ok Checking for prepared transactions ok Checking for reg* data types in user tables ok Checking for contrib/isn with bigint-passing mismatch ok Checking for tables WITH OIDS ok Checking for invalid "sql_identifier" user columns ok Checking for presence of required libraries ok Checking database user is the install user ok Checking for prepared transactions ok *Clusters are compatible* But if I execute the same command right after that without the "--check" flag I get: Performing Consistency Checks ----------------------------- Checking cluster versions ok The target cluster was not shut down cleanly. Failure, exiting Both of the servers were still shut down. If I start form the beginning with a fresh Prostgres 12 installation and execute the upgrade command right away without the "--check" flag I get the problem described in my original post. Best regards, TalGlo -- Sent from: https://www.postgresql-archive.org/PostgreSQL-general-f1843780.html
Re: Upgrade Process Says "The database server was not shut downcleanly" but it was
От
Adrian Klaver
Дата:
On 5/11/20 2:45 PM, TalGloz wrote: > Well I tried your steps, both Postgres 10 and 12 are in perfect running > conditions and were shut down using the "systemctl" command. When I execute: > > /usr/pgsql-12/bin/pg_upgrade --old-bindir=/usr/pgsql-10/bin/ > --new-bindir=/usr/pgsql-12/bin/ --old-datadir=/var/lib/pgsql/10/data > --new-datadir=/var/lib/pgsql/12/data --old-options '-c > config_file=/var/lib/pgsql/10/data/postgresql.conf' --new-options '-c > config_file=/var/lib/pgsql/12/data/postgresql.conf' --check The --*-options are redundant as you have already pointed pg_upgrade at them with --*-datadir. Not sure it has anything to do with this, but you might want to take them out just to be sure. Have you run: ps ax| grep post before and between steps to make sure there is not a Postgres instance running? > > I get: > > Performing Consistency Checks > ----------------------------- > Checking cluster versions ok > Checking database user is the install user ok > Checking database connection settings ok > Checking for prepared transactions ok > Checking for reg* data types in user tables ok > Checking for contrib/isn with bigint-passing mismatch ok > Checking for tables WITH OIDS ok > Checking for invalid "sql_identifier" user columns ok > Checking for presence of required libraries ok > Checking database user is the install user ok > Checking for prepared transactions ok > > *Clusters are compatible* > > But if I execute the same command right after that without the "--check" > flag I get: > > Performing Consistency Checks > ----------------------------- > Checking cluster versions ok > > The target cluster was not shut down cleanly. > Failure, exiting > > > Both of the servers were still shut down. If I start form the beginning with > a fresh Prostgres 12 installation and execute the upgrade command right away > without the "--check" flag I get the problem described in my original post. > > Best regards, > TalGlo > > > > -- > Sent from: https://www.postgresql-archive.org/PostgreSQL-general-f1843780.html > > -- Adrian Klaver adrian.klaver@aklaver.com
Re: Upgrade Process Says "The database server was not shut downcleanly" but it was
От
Adrian Klaver
Дата:
On 5/11/20 2:45 PM, TalGloz wrote: > Well I tried your steps, both Postgres 10 and 12 are in perfect running > conditions and were shut down using the "systemctl" command. When I execute: > > /usr/pgsql-12/bin/pg_upgrade --old-bindir=/usr/pgsql-10/bin/ > --new-bindir=/usr/pgsql-12/bin/ --old-datadir=/var/lib/pgsql/10/data > --new-datadir=/var/lib/pgsql/12/data --old-options '-c > config_file=/var/lib/pgsql/10/data/postgresql.conf' --new-options '-c > config_file=/var/lib/pgsql/12/data/postgresql.conf' --check > > I get: > > Performing Consistency Checks > ----------------------------- > Checking cluster versions ok > Checking database user is the install user ok > Checking database connection settings ok > Checking for prepared transactions ok > Checking for reg* data types in user tables ok > Checking for contrib/isn with bigint-passing mismatch ok > Checking for tables WITH OIDS ok > Checking for invalid "sql_identifier" user columns ok > Checking for presence of required libraries ok > Checking database user is the install user ok > Checking for prepared transactions ok > > *Clusters are compatible* > > But if I execute the same command right after that without the "--check" > flag I get: > > Performing Consistency Checks > ----------------------------- > Checking cluster versions ok > > The target cluster was not shut down cleanly. > Failure, exiting > > > Both of the servers were still shut down. If I start form the beginning with > a fresh Prostgres 12 installation and execute the upgrade command right away > without the "--check" flag I get the problem described in my original post. How are you installing Postgres? Are the 10 and 12 packages coming from the same location? > > Best regards, > TalGlo > > > > -- > Sent from: https://www.postgresql-archive.org/PostgreSQL-general-f1843780.html > > -- Adrian Klaver adrian.klaver@aklaver.com
Adrian Klaver-4 wrote > On 5/11/20 2:45 PM, TalGloz wrote: >> Well I tried your steps, both Postgres 10 and 12 are in perfect running >> conditions and were shut down using the "systemctl" command. When I >> execute: >> >> /usr/pgsql-12/bin/pg_upgrade --old-bindir=/usr/pgsql-10/bin/ >> --new-bindir=/usr/pgsql-12/bin/ --old-datadir=/var/lib/pgsql/10/data >> --new-datadir=/var/lib/pgsql/12/data --old-options '-c >> config_file=/var/lib/pgsql/10/data/postgresql.conf' --new-options '-c >> config_file=/var/lib/pgsql/12/data/postgresql.conf' --check >> >> I get: >> >> Performing Consistency Checks >> ----------------------------- >> Checking cluster versions >> ok >> Checking database user is the install user ok >> Checking database connection settings ok >> Checking for prepared transactions ok >> Checking for reg* data types in user tables ok >> Checking for contrib/isn with bigint-passing mismatch ok >> Checking for tables WITH OIDS ok >> Checking for invalid "sql_identifier" user columns ok >> Checking for presence of required libraries ok >> Checking database user is the install user ok >> Checking for prepared transactions ok >> >> *Clusters are compatible* >> >> But if I execute the same command right after that without the "--check" >> flag I get: >> >> Performing Consistency Checks >> ----------------------------- >> Checking cluster versions ok >> >> The target cluster was not shut down cleanly. >> Failure, exiting >> >> >> Both of the servers were still shut down. If I start form the beginning >> with >> a fresh Prostgres 12 installation and execute the upgrade command right >> away >> without the "--check" flag I get the problem described in my original >> post. > > How are you installing Postgres? > > Are the 10 and 12 packages coming from the same location? > >> >> Best regards, >> TalGlo >> >> >> >> -- >> Sent from: >> https://www.postgresql-archive.org/PostgreSQL-general-f1843780.html >> >> > > > -- > Adrian Klaver > adrian.klaver@ This is my output for "ps ax| grep post" it is always the same after shunting down the services. 11232 pts/0 S+ 0:00 grep --color=auto post I've used the same installation steps for both of the instances: 1. Add the PostgreSQL Yum Repository (needed to be done once) 2. Install the desired Postgres instance using "sudo dnf install postgresqlXX-*" Best regards, TalGloz -- Sent from: https://www.postgresql-archive.org/PostgreSQL-general-f1843780.html
Re: Upgrade Process Says "The database server was not shut downcleanly" but it was
От
Adrian Klaver
Дата:
On 5/12/20 2:51 PM, TalGloz wrote: > Adrian Klaver-4 wrote >> On 5/11/20 2:45 PM, TalGloz wrote: >>> Well I tried your steps, both Postgres 10 and 12 are in perfect running >>> conditions and were shut down using the "systemctl" command. When I >>> execute: >>> >>> /usr/pgsql-12/bin/pg_upgrade --old-bindir=/usr/pgsql-10/bin/ >>> --new-bindir=/usr/pgsql-12/bin/ --old-datadir=/var/lib/pgsql/10/data >>> --new-datadir=/var/lib/pgsql/12/data --old-options '-c >>> config_file=/var/lib/pgsql/10/data/postgresql.conf' --new-options '-c >>> config_file=/var/lib/pgsql/12/data/postgresql.conf' --check >>> >>> I get: >>> >>> Performing Consistency Checks >>> ----------------------------- >>> Checking cluster versions >>> ok >>> Checking database user is the install user ok >>> Checking database connection settings ok >>> Checking for prepared transactions ok >>> Checking for reg* data types in user tables ok >>> Checking for contrib/isn with bigint-passing mismatch ok >>> Checking for tables WITH OIDS ok >>> Checking for invalid "sql_identifier" user columns ok >>> Checking for presence of required libraries ok >>> Checking database user is the install user ok >>> Checking for prepared transactions ok >>> >>> *Clusters are compatible* >>> >>> But if I execute the same command right after that without the "--check" >>> flag I get: >>> >>> Performing Consistency Checks >>> ----------------------------- >>> Checking cluster versions ok >>> >>> The target cluster was not shut down cleanly. >>> Failure, exiting >>> >>> >>> Both of the servers were still shut down. If I start form the beginning >>> with >>> a fresh Prostgres 12 installation and execute the upgrade command right >>> away >>> without the "--check" flag I get the problem described in my original >>> post. >> >> How are you installing Postgres? >> >> Are the 10 and 12 packages coming from the same location? >> >>> >>> Best regards, >>> TalGlo >>> >>> >>> >>> -- >>> Sent from: >>> https://www.postgresql-archive.org/PostgreSQL-general-f1843780.html >>> >>> >> >> >> -- >> Adrian Klaver > >> adrian.klaver@ > > > > This is my output for "ps ax| grep post" it is always the same after > shunting down the services. > 11232 pts/0 S+ 0:00 grep --color=auto post > > I've used the same installation steps for both of the instances: > > 1. Add the PostgreSQL Yum Repository (needed to be done once) > 2. Install the desired Postgres instance using "sudo dnf install > postgresqlXX-*" Have you tried the upgrade w/o the --options switches? Do the regular Postgres logs show anything? Have the systemctl scripts been changed? And wild hunch from another thread: "Is there a significant difference in Memory settings between 9.5 and 12" In particular this message: https://www.postgresql.org/message-id/CAEaSS0YDpb_L1Ve%2B4aNhNz7nSN5M%2BEpT9fV8AUuEYnY2tDtuDw%40mail.gmail.com See bottom of message for what OP says about sysctl. > > Best regards, > TalGloz > > > > -- > Sent from: https://www.postgresql-archive.org/PostgreSQL-general-f1843780.html > > -- Adrian Klaver adrian.klaver@aklaver.com