Обсуждение: 7.4 <-> 8.0
All, Couple questions from to a recent upgrade from 7.4.8 to 8.0.3. With servers for both versions running on the same machine, pg_dumpall -p 5432 -c | psql -p 5433 template1 appears to have migrated the databases smoothly. At least I didn't notice any errors in the process, and all the data appears to be there. The one troubling thing is the new 8.0 databases take about half the disk space as the 7.4 databases. Similar dump | psql to recreate the databases on other machines running 7.4.7 and 7.4.8 show the same thing - the fresh copies take half the space of the old. What might be happening with the old db, so it takes so much more space? Could half the data be missing? What's the best way to move data from 8.0 to 7.4? The 8.0 pg_dump writes a dump that doesn't restore with 7.4 psql, and 7.4 pg_dump doesn't seem to handle an 8.0 database. At least I haven't found the right switches... Thanks, warren
Hi Warren
On the space issue, seems at a first sight that the old db needs a vacuum full to free some room.
Try to identify your biggest tables, isolate the vacuum against them, and then escalate to the whole db, with a vacuum full.
Thinking that you should double check the vacuum documentation, to understand better which are the effects of the different vacuum modes. This will clear out your doubts on why such amount of space is being allocated, but not being used.
My two cents there.
Best wishes,
Guido.
On 9/15/05, Warren Snelling <snelling@email.marc.usda.gov> wrote:
All,
Couple questions from to a recent upgrade from 7.4.8 to 8.0.3. With
servers for both versions running on the same machine,
pg_dumpall -p 5432 -c | psql -p 5433 template1
appears to have migrated the databases smoothly. At least I didn't
notice any errors in the process, and all the data appears to be there.
The one troubling thing is the new 8.0 databases take about half the
disk space as the 7.4 databases. Similar dump | psql to recreate the
databases on other machines running 7.4.7 and 7.4.8 show the same thing
- the fresh copies take half the space of the old. What might be
happening with the old db, so it takes so much more space? Could half
the data be missing?
What's the best way to move data from 8.0 to 7.4? The 8.0 pg_dump
writes a dump that doesn't restore with 7.4 psql, and 7.4 pg_dump
doesn't seem to handle an 8.0 database. At least I haven't found the
right switches...
Thanks,
warren
---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster
--
"Adopting the position that you are smarter than an automatic
optimization algorithm is generally a good way to achieve less
performance, not more" - Tom Lane.
still thinking on this topic, read about the REINDEX command, that will recreate your indexes, per table or per database.
Regards,
Guido.
On 9/15/05, Guido Barosio <gbarosio@gmail.com> wrote:
Hi WarrenOn the space issue, seems at a first sight that the old db needs a vacuum full to free some room.Try to identify your biggest tables, isolate the vacuum against them, and then escalate to the whole db, with a vacuum full.Thinking that you should double check the vacuum documentation, to understand better which are the effects of the different vacuum modes. This will clear out your doubts on why such amount of space is being allocated, but not being used.My two cents there.Best wishes,Guido.
--On 9/15/05, Warren Snelling <snelling@email.marc.usda.gov > wrote:All,
Couple questions from to a recent upgrade from 7.4.8 to 8.0.3. With
servers for both versions running on the same machine,
pg_dumpall -p 5432 -c | psql -p 5433 template1
appears to have migrated the databases smoothly. At least I didn't
notice any errors in the process, and all the data appears to be there.
The one troubling thing is the new 8.0 databases take about half the
disk space as the 7.4 databases. Similar dump | psql to recreate the
databases on other machines running 7.4.7 and 7.4.8 show the same thing
- the fresh copies take half the space of the old. What might be
happening with the old db, so it takes so much more space? Could half
the data be missing?
What's the best way to move data from 8.0 to 7.4? The 8.0 pg_dump
writes a dump that doesn't restore with 7.4 psql, and 7.4 pg_dump
doesn't seem to handle an 8.0 database. At least I haven't found the
right switches...
Thanks,
warren
---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster
"Adopting the position that you are smarter than an automatic
optimization algorithm is generally a good way to achieve less
performance, not more" - Tom Lane.
--
"Adopting the position that you are smarter than an automatic
optimization algorithm is generally a good way to achieve less
performance, not more" - Tom Lane.
Guido, Thanks for the pointers. I have routinely vacuumed the db, but not using full. By itself, the "vacuum full" run overnight did not free the space I was expecting - du still showed the old database taking about 12 Gb disk, and the fresh databases taking about 6.5 Gb. Running "reindex" on one large table freed ~5 Gb, so the old db now takes 7 Gb. Reindexing the remaining tables should get the databases to the same size. Would you suggest scripting the vacuum full / reindex process to run periodically (once a week or so)? Right now disk space on this machine is not an issue, but vacuuming and reindexing occasionally should keep the db from consuming too much excess space. Thanks again, warren > still thinking on this topic, read about the REINDEX command, that > will recreate your indexes, per table or per database. > > Regards, > Guido. > > > On 9/15/05, Guido Barosio <gbarosio@gmail.com> wrote: > Hi Warren > > On the space issue, seems at a first sight that the old db > needs a vacuum full to free some room. > Try to identify your biggest tables, isolate the vacuum > against them, and then escalate to the whole db, with a vacuum > full. > > Thinking that you should double check the vacuum > documentation, to understand better which are the effects of > the different vacuum modes. This will clear out your doubts on > why such amount of space is being allocated, but not being > used. > > My two cents there. > > Best wishes, > Guido. > > On 9/15/05, Warren Snelling <snelling@email.marc.usda.gov> > wrote: > All, > > Couple questions from to a recent upgrade from 7.4.8 > to 8.0.3. With > servers for both versions running on the same > machine, > > pg_dumpall -p 5432 -c | psql -p 5433 template1 > > appears to have migrated the databases smoothly. At > least I didn't > notice any errors in the process, and all the data > appears to be there. > > The one troubling thing is the new 8.0 databases take > about half the > disk space as the 7.4 databases. Similar dump | psql > to recreate the > databases on other machines running 7.4.7 and 7.4.8 > show the same thing > - the fresh copies take half the space of the > old. What might be > happening with the old db, so it takes so much more > space? Could half > the data be missing? > > What's the best way to move data from 8.0 to 7.4? The > 8.0 pg_dump > writes a dump that doesn't restore with 7.4 psql, and > 7.4 pg_dump > doesn't seem to handle an 8.0 database. At least I > haven't found the > right switches... > > Thanks, > warren > > > ---------------------------(end of > broadcast)--------------------------- > TIP 2: Don't 'kill -9' the postmaster > > > > > -- > "Adopting the position that you are smarter than an automatic > optimization algorithm is generally a good way to achieve less > performance, not more" - Tom Lane. > > > > -- > "Adopting the position that you are smarter than an automatic > optimization algorithm is generally a good way to achieve less > performance, not more" - Tom Lane.
Hi Warren,
Actually, in my particular case (old pgsql version, and a big db), reindex and vacuum full are extremely expensive in operation costs,
and I am running them myself on planned conditions. But that's my case. Scripting this, with a cron hand,
would be ok, and I guess that is something you will prolly find out there.
The reindex process is something to watch at, and not to leave running during the night, without some bird eye on it,
at least on my scenery.
Best wishes,
Guido.
--
"Adopting the position that you are smarter than an automatic
optimization algorithm is generally a good way to achieve less
performance, not more" - Tom Lane.
Actually, in my particular case (old pgsql version, and a big db), reindex and vacuum full are extremely expensive in operation costs,
and I am running them myself on planned conditions. But that's my case. Scripting this, with a cron hand,
would be ok, and I guess that is something you will prolly find out there.
The reindex process is something to watch at, and not to leave running during the night, without some bird eye on it,
at least on my scenery.
Best wishes,
Guido.
On 9/16/05, Warren Snelling <snelling@email.marc.usda.gov> wrote:
Guido,
Thanks for the pointers. I have routinely vacuumed the db, but not
using full.
By itself, the "vacuum full" run overnight did not free the space I was
expecting - du still showed the old database taking about 12 Gb disk,
and the fresh databases taking about 6.5 Gb. Running "reindex" on one
large table freed ~5 Gb, so the old db now takes 7 Gb. Reindexing the
remaining tables should get the databases to the same size.
Would you suggest scripting the vacuum full / reindex process to run
periodically (once a week or so)? Right now disk space on this machine
is not an issue, but vacuuming and reindexing occasionally should keep
the db from consuming too much excess space.
Thanks again,
warren
> still thinking on this topic, read about the REINDEX command, that
> will recreate your indexes, per table or per database.
>
> Regards,
> Guido.
>
>
> On 9/15/05, Guido Barosio <gbarosio@gmail.com> wrote:
> Hi Warren
>
> On the space issue, seems at a first sight that the old db
> needs a vacuum full to free some room.
> Try to identify your biggest tables, isolate the vacuum
> against them, and then escalate to the whole db, with a vacuum
> full.
>
> Thinking that you should double check the vacuum
> documentation, to understand better which are the effects of
> the different vacuum modes. This will clear out your doubts on
> why such amount of space is being allocated, but not being
> used.
>
> My two cents there.
>
> Best wishes,
> Guido.
>
> On 9/15/05, Warren Snelling < snelling@email.marc.usda.gov>
> wrote:
> All,
>
> Couple questions from to a recent upgrade from 7.4.8
> to 8.0.3. With
> servers for both versions running on the same
> machine,
>
> pg_dumpall -p 5432 -c | psql -p 5433 template1
>
> appears to have migrated the databases smoothly. At
> least I didn't
> notice any errors in the process, and all the data
> appears to be there.
>
> The one troubling thing is the new 8.0 databases take
> about half the
> disk space as the 7.4 databases. Similar dump | psql
> to recreate the
> databases on other machines running 7.4.7 and 7.4.8
> show the same thing
> - the fresh copies take half the space of the
> old. What might be
> happening with the old db, so it takes so much more
> space? Could half
> the data be missing?
>
> What's the best way to move data from 8.0 to 7.4? The
> 8.0 pg_dump
> writes a dump that doesn't restore with 7.4 psql, and
> 7.4 pg_dump
> doesn't seem to handle an 8.0 database. At least I
> haven't found the
> right switches...
>
> Thanks,
> warren
>
>
> ---------------------------(end of
> broadcast)---------------------------
> TIP 2: Don't 'kill -9' the postmaster
>
>
>
>
> --
> "Adopting the position that you are smarter than an automatic
> optimization algorithm is generally a good way to achieve less
> performance, not more" - Tom Lane.
>
>
>
> --
> "Adopting the position that you are smarter than an automatic
> optimization algorithm is generally a good way to achieve less
> performance, not more" - Tom Lane.
--
"Adopting the position that you are smarter than an automatic
optimization algorithm is generally a good way to achieve less
performance, not more" - Tom Lane.
On Fri, 2005-09-16 at 10:57, Warren Snelling wrote: > Guido, > > Thanks for the pointers. I have routinely vacuumed the db, but not > using full. > > By itself, the "vacuum full" run overnight did not free the space I was > expecting - du still showed the old database taking about 12 Gb disk, > and the fresh databases taking about 6.5 Gb. Running "reindex" on one > large table freed ~5 Gb, so the old db now takes 7 Gb. Reindexing the > remaining tables should get the databases to the same size. > > Would you suggest scripting the vacuum full / reindex process to run > periodically (once a week or so)? Right now disk space on this machine > is not an issue, but vacuuming and reindexing occasionally should keep > the db from consuming too much excess space. Assuming that you run regular vacuums often enough, and your fsm settings are high enough to hold on to all the references to the deleted tuples, you shouldn't need to run either full vacuums or reindexes. There are some usage patterns that can result in indexes with very sparsely populated blocks, such as when you create indexes to the numbers 1 to 1000, then delete 95% or so of those references, and all the new ones will be >1000, then repeat, increasing the number all the time. But that type of situation is the only that might require occasional reindexing.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Warren Snelling wrote:
>
>
> Would you suggest scripting the vacuum full / reindex process to
> run periodically (once a week or so)? Right now disk space on this
> machine is not an issue, but vacuuming and reindexing occasionally
> should keep the db from consuming too much excess space.
>
7.4 has a problem with vacuuming indexes I seam to remember. I
think thats this is one of the fixes in 8.0. Apart from anything a
large index is going to slow your database down as the database has to
scan past junk everytime it does an index lookup.
Peter Childs
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
iD8DBQFDKvMKJWgkOaNnHdMRAsBEAKCv3ZWdy1a4+qQG0U9Ov85TPCembQCfazPH
QVPj5xEDPSumy1Z4FUSTgf4=
=As1l
-----END PGP SIGNATURE-----
---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?
http://archives.postgresql.org
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Warren Snelling wrote:
>
>
> Would you suggest scripting the vacuum full / reindex process to
> run periodically (once a week or so)? Right now disk space on this
> machine is not an issue, but vacuuming and reindexing occasionally
> should keep the db from consuming too much excess space.
>
7.4 has a problem with vacuuming indexes I seam to remember. I
think thats this is one of the fixes in 8.0. Apart from anything a
large index is going to slow your database down as the database has to
scan past junk everytime it does an index lookup.
Peter Childs
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
iD8DBQFDKvMKJWgkOaNnHdMRAsBEAKCv3ZWdy1a4+qQG0U9Ov85TPCembQCfazPH
QVPj5xEDPSumy1Z4FUSTgf4=
=As1l
-----END PGP SIGNATURE-----
---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?
http://archives.postgresql.org