Обсуждение: [ADMIN] upgrade path from PG 8.3 to PG 9.5

Hi List,
we still have 2 hosts  running the old PG 8.3  version, we finally
agreed to move to PG9.5, please advise what will be  the process for
this upgrade ? the db size  for upgrade is 500GB(  with 4 table spaces
on # volumes)
( do I need to move first to PG9.0 and next move up?)
Isabella

> Hi List,
> we still have 2 hosts  running the old PG 8.3  version, we finally
> agreed to move to PG9.5, please advise what will be  the process for
> this upgrade ? the db size  for upgrade is 500GB(  with 4 table spaces
> on # volumes)
> ( do I need to move first to PG9.0 and next move up?)
> Isabella
>

No, you need.

Use pgupgrade

> --
> Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-admin
>


--
Saludos,
Gilberto Castillo
ETECSA, La Habana, Cuba

Guten Tag Keith,
am Donnerstag, 23. Februar 2017 um 18:57 schrieben Sie:

> You can go directly from 8.3 to 9.5. Just be sure and use the
> pg_dump from 9.5 to generate the dump file from 8.3.

Is there a specific reason for that? My current approach for upgrades
was dumping the old database using the old pg_dump, uninstalling the
old Postgres, installing the new one and restore from the old dump.
That way I didn't accidently mix-up installations or use wrong tools
or such.

Did pg_dump have bugs in the past which resulted in pg_restore not
being able to read the dumps?

Mit freundlichen Grüßen,

Thorsten Schöning

--
Thorsten Schöning       E-Mail: Thorsten.Schoening@AM-SoFT.de
AM-SoFT IT-Systeme      http://www.AM-SoFT.de/

Telefon...........05151-  9468- 55
Fax...............05151-  9468- 88
Mobil..............0178-8 9468- 04

AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln
AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow

Greetings,

* Thorsten Schöning (tschoening@am-soft.de) wrote:
> Guten Tag Keith,
> am Donnerstag, 23. Februar 2017 um 18:57 schrieben Sie:
>
> > You can go directly from 8.3 to 9.5. Just be sure and use the
> > pg_dump from 9.5 to generate the dump file from 8.3.
>
> Is there a specific reason for that? My current approach for upgrades
> was dumping the old database using the old pg_dump, uninstalling the
> old Postgres, installing the new one and restore from the old dump.

Yes.  Using the older pg_dump may result in dumps that can't be restored
into the newer version of PG because things like keywords have been
added and must now be quoted.  There are possibly other things that have
been changed between the old version and the new one which would also
require the new pg_dump to be used, but keywords are the big one.

> That way I didn't accidently mix-up installations or use wrong tools
> or such.

Having a good, clean, way to install and handle multiple versions
concurrently is valuable not just for pg_dump but also for pg_upgrade
and just generally a good thing.  The PGDG RPM packages do an alright
job of this, and the Debian packaging is quite good at handling it.

> Did pg_dump have bugs in the past which resulted in pg_restore not
> being able to read the dumps?

This isn't generally an issue, but what goes into the dump file is
straight SQL for things like creating tables and such; pg_restore does
not do anything for dealing with keyword changes or anything like that.

Thanks!

Stephen

Greetings,

* David G. Johnston (david.g.johnston@gmail.com) wrote:
> On Tue, Mar 21, 2017 at 9:40 AM, Stephen Frost <sfrost@snowman.net> wrote:
> > * Thorsten Schöning (tschoening@am-soft.de) wrote:
> > > Guten Tag Keith,
> > > am Donnerstag, 23. Februar 2017 um 18:57 schrieben Sie:
> > >
> > > > You can go directly from 8.3 to 9.5. Just be sure and use the
> > > > pg_dump from 9.5 to generate the dump file from 8.3.
> > >
> > > Is there a specific reason for that? My current approach for upgrades
> > > was dumping the old database using the old pg_dump, uninstalling the
> > > old Postgres, installing the new one and restore from the old dump.
> >
> > Yes.  Using the older pg_dump may result in dumps that can't be restored
> > into the newer version of PG because things like keywords have been
> > added and must now be quoted.  There are possibly other things that have
> > been changed between the old version and the new one which would also
> > require the new pg_dump to be used, but keywords are the big one.

> The proper solution to the keyword/identifier quoting problem is to use
> "--quote-all-identifiers".

Using the version of pg_dump to which you are upgrading is an entirely
supported and 'proper' approach to dealing with that issue.  Using
--quote-all-identifiers, in my opinion anyway, leads to rather ugly
results.

As I also mention, there are other things in newer versions that pg_dump
does its best to address (including things like checking for role names
in older versions starting with "pg_", which is not allowed in 9.6+).
Using the pg_dump from the version of PG to which you are upgrading is
the correct and supported approach to doing upgrades.  Using
--quote-all-identifiers is not.

Thanks!

Stephen

On Tue, Mar 21, 2017 at 10:54 AM, Stephen Frost <sfrost@snowman.net> wrote:
> Greetings,
>
> * David G. Johnston (david.g.johnston@gmail.com) wrote:
>> On Tue, Mar 21, 2017 at 9:40 AM, Stephen Frost <sfrost@snowman.net> wrote:
>> > * Thorsten Schöning (tschoening@am-soft.de) wrote:
>> > > Guten Tag Keith,
>> > > am Donnerstag, 23. Februar 2017 um 18:57 schrieben Sie:
>> > >
>> > > > You can go directly from 8.3 to 9.5. Just be sure and use the
>> > > > pg_dump from 9.5 to generate the dump file from 8.3.
>> > >
>> > > Is there a specific reason for that? My current approach for upgrades
>> > > was dumping the old database using the old pg_dump, uninstalling the
>> > > old Postgres, installing the new one and restore from the old dump.
>> >
>> > Yes.  Using the older pg_dump may result in dumps that can't be restored
>> > into the newer version of PG because things like keywords have been
>> > added and must now be quoted.  There are possibly other things that have
>> > been changed between the old version and the new one which would also
>> > require the new pg_dump to be used, but keywords are the big one.
>
>> The proper solution to the keyword/identifier quoting problem is to use
>> "--quote-all-identifiers".
>
> Using the version of pg_dump to which you are upgrading is an entirely
> supported and 'proper' approach to dealing with that issue.  Using
> --quote-all-identifiers, in my opinion anyway, leads to rather ugly
> results.
>
> As I also mention, there are other things in newer versions that pg_dump
> does its best to address (including things like checking for role names
> in older versions starting with "pg_", which is not allowed in 9.6+).
> Using the pg_dump from the version of PG to which you are upgrading is
> the correct and supported approach to doing upgrades.  Using
> --quote-all-identifiers is not.

Yeah there are plenty of little road bumps to hit especially when
going from something  as old as 8.3 to 9.5. emcoding issues pop up etc
etc.

Using the new version to dump is THE way to go if you are gonna dump restore.

IF you need the minimal amount of downtime possible, the only
reasonable solution is some form of logical replication like slony or
longdiste etc.

According to this page: http://slony.info/documentation/requirements.html

slony 2.2.5 supports versions from 8.3 to 9.5 so you can upgrade from
one to the other in one fell swoop and your downtime will be measured
in seconds, or minutes at most. I suggest create a pair of test
servers running 8.3 and 9.5 to test the migration on ahead of time.
If you don't need to keep downtime to a minimum dump/restore is WAY
easier and you can do a test run while the main server is running just
by taking a backup and see if it restores cleanly and properly to the
new 9.5 machine. If you do want minimal downtime, learning slony or
some other logical replication system is worth your time. Slony docs
have a fairly old but still mostly accurate how to page here:
http://slony.info/documentation/1.2/versionupgrade.html

--
To understand recursion, one must first understand recursion.

"David G. Johnston" <david.g.johnston@gmail.com> writes:
> On Tue, Mar 21, 2017 at 9:40 AM, Stephen Frost <sfrost@snowman.net> wrote:
>> Yes.  Using the older pg_dump may result in dumps that can't be restored
>> into the newer version of PG because things like keywords have been
>> added and must now be quoted.  There are possibly other things that have
>> been changed between the old version and the new one which would also
>> require the new pg_dump to be used, but keywords are the big one.

> The proper solution to the keyword/identifier quoting problem is to use
> "--quote-all-identifiers"​.

... which didn't exist before 9.1 ...

            regards, tom lane

On 21/03/2017 21:00, Scott Marlowe wrote:
> On Tue, Mar 21, 2017 at 10:54 AM, Stephen Frost <sfrost@snowman.net> wrote:
>> Greetings,
>>
>> * David G. Johnston (david.g.johnston@gmail.com) wrote:
>>> On Tue, Mar 21, 2017 at 9:40 AM, Stephen Frost <sfrost@snowman.net> wrote:
>>>> * Thorsten Schöning (tschoening@am-soft.de) wrote:
>>>>> Guten Tag Keith,
>>>>> am Donnerstag, 23. Februar 2017 um 18:57 schrieben Sie:
>>>>>
>>>>>> You can go directly from 8.3 to 9.5. Just be sure and use the
>>>>>> pg_dump from 9.5 to generate the dump file from 8.3.
>>>>> Is there a specific reason for that? My current approach for upgrades
>>>>> was dumping the old database using the old pg_dump, uninstalling the
>>>>> old Postgres, installing the new one and restore from the old dump.
>>>> Yes.  Using the older pg_dump may result in dumps that can't be restored
>>>> into the newer version of PG because things like keywords have been
>>>> added and must now be quoted.  There are possibly other things that have
>>>> been changed between the old version and the new one which would also
>>>> require the new pg_dump to be used, but keywords are the big one.
>>> The proper solution to the keyword/identifier quoting problem is to use
>>> "--quote-all-identifiers".
>> Using the version of pg_dump to which you are upgrading is an entirely
>> supported and 'proper' approach to dealing with that issue.  Using
>> --quote-all-identifiers, in my opinion anyway, leads to rather ugly
>> results.
>>
>> As I also mention, there are other things in newer versions that pg_dump
>> does its best to address (including things like checking for role names
>> in older versions starting with "pg_", which is not allowed in 9.6+).
>> Using the pg_dump from the version of PG to which you are upgrading is
>> the correct and supported approach to doing upgrades.  Using
>> --quote-all-identifiers is not.
> Yeah there are plenty of little road bumps to hit especially when
> going from something  as old as 8.3 to 9.5. emcoding issues pop up etc
> etc.
>
> Using the new version to dump is THE way to go if you are gonna dump restore.
>
> IF you need the minimal amount of downtime possible, the only
> reasonable solution is some form of logical replication like slony or
> longdiste etc.
We used pg_upgrade recently for migrating from 8.3 -> 9.4 (the last version which supports 8.3), and it went nice. We
hadsome custom functions in C, and pg_upgrade is a disciplined way to deal with  
the migration.
We will deploy this migration procedure to over 100 servers running 8.3.
> According to this page: http://slony.info/documentation/requirements.html
>
> slony 2.2.5 supports versions from 8.3 to 9.5 so you can upgrade from
> one to the other in one fell swoop and your downtime will be measured
> in seconds, or minutes at most. I suggest create a pair of test
> servers running 8.3 and 9.5 to test the migration on ahead of time.
> If you don't need to keep downtime to a minimum dump/restore is WAY
> easier and you can do a test run while the main server is running just
> by taking a backup and see if it restores cleanly and properly to the
> new 9.5 machine. If you do want minimal downtime, learning slony or
> some other logical replication system is worth your time. Slony docs
> have a fairly old but still mostly accurate how to page here:
> http://slony.info/documentation/1.2/versionupgrade.html
>


--
Achilleas Mantzios
IT DEV Lead
IT DEPT
Dynacom Tankers Mgmt

We used pg_upgrade recently for migrating from 8.3 -> 9.4 (the last version which supports 8.3), and it went nice. We had some custom functions in C, and pg_upgrade is a disciplined way to deal with the migration

I would be curious how that worked since the documentation states the old version must be at least 8.4.7 for pg_upgrade to work.

https://www.postgresql.org/docs/9.6/static/pgupgrade.html

Just checked this with 8.3.22 and confirmed the binary won't even let you run it.

$ pg_upgrade -b /opt/pgsql8323/bin -B /opt/pgsql962/bin -c -d /opt/pgdata/83 -D /opt/pgdata/83upgrade
Performing Consistency Checks
-----------------------------
Checking cluster versions                                  
This utility can only upgrade from PostgreSQL version 8.4 and later.
Failure, exiting

 

Not trying to be pedantic, but don't want to lead someone down a road that won't work properly.

Keith--
Achilleas Mantzios
IT DEV Lead
IT DEPT
Dynacom Tankers Mgmt

--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin

-- 
Achilleas Mantzios
IT DEV Lead
IT DEPT
Dynacom Tankers Mgmt