Обсуждение: File system backup question
Benjamin Krajmalnik wrote:
> I am currently running PG 8.1.4 windows.
> The system is for a real time monitoring application (so downtime needs
> to be minimized if at all possible).
>
> I will be migratng to a new server running PG 8.1.4 on FreeBSD 6.1.
> I have been playing with various options for migrating the data. The
> database is currently about 8.5 GB.
> pg_dump took about 90 minutes.
> pg_restore has been running for over an hour and is abou 4% done (based
> on row counts on the tables).
>
> I used the pg_dump option to insert records (as oposed to use the copy
> command - I don't know which would be faster for the restore).
That is why it is taking so long.
Do not use inserts
Turn off fsync
Crank up shared buffers
make sure you have lots of checkpoint_segments.
Joshua D. Drake
>
> In any case, I was thinking about performing a file system backup.
> My concern is that I am movinf between different OS's (although both are
> i386 architecture). I will probably try it tomorrow to see if it works,
> but was just wondering if there are any caveats which I should be aware
> of.
> I hope this works, since for my scenario, this will have the least
> downtime.
>
> Thanks in advance for any advice.
>
> Regards,
>
> Benjamin
>
>
>
> PS. For those who have asked about performance, Windows is much slower
> than FreeBSD.
>
> Machine specs are as follows:
>
> Windows 2003 server, HP DL360, SCSI RAID-1, 2GB RAM, 2.8GHz XEON HT
> FreeBSD 6.1, SuperMicro 5014C-T, SATA RAID-1, 1GB RAM, P4 3GHz
>
> My stored procedure call went from 47ms avg to 6 ms average execution
> time on server, as reported by turning logging on
> All queries in general appeared to run between 5x to 10x faster.
>
>
>
>
>
--
=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/
I believe I have to use onserts due to the problem with the records with the improper utf encoding. According to the docs this will cause the restore to fail.
I will try it as well as a file system restore.
-----Original Message-----
From: Joshua D. Drake <jd@commandprompt.com>
To: Benjamin Krajmalnik <kraj@illumen.com>
CC: pgsql-admin@postgresql.org <pgsql-admin@postgresql.org>
Sent: Thu Aug 03 21:01:54 2006
Subject: Re: [ADMIN] File system backup question
Benjamin Krajmalnik wrote:
> I am currently running PG 8.1.4 windows.
> The system is for a real time monitoring application (so downtime needs
> to be minimized if at all possible).
>
> I will be migratng to a new server running PG 8.1.4 on FreeBSD 6.1.
> I have been playing with various options for migrating the data. The
> database is currently about 8.5 GB.
> pg_dump took about 90 minutes.
> pg_restore has been running for over an hour and is abou 4% done (based
> on row counts on the tables).
>
> I used the pg_dump option to insert records (as oposed to use the copy
> command - I don't know which would be faster for the restore).
That is why it is taking so long.
Do not use inserts
Turn off fsync
Crank up shared buffers
make sure you have lots of checkpoint_segments.
Joshua D. Drake
>
> In any case, I was thinking about performing a file system backup.
> My concern is that I am movinf between different OS's (although both are
> i386 architecture). I will probably try it tomorrow to see if it works,
> but was just wondering if there are any caveats which I should be aware
> of.
> I hope this works, since for my scenario, this will have the least
> downtime.
>
> Thanks in advance for any advice.
>
> Regards,
>
> Benjamin
>
>
>
> PS. For those who have asked about performance, Windows is much slower
> than FreeBSD.
>
> Machine specs are as follows:
>
> Windows 2003 server, HP DL360, SCSI RAID-1, 2GB RAM, 2.8GHz XEON HT
> FreeBSD 6.1, SuperMicro 5014C-T, SATA RAID-1, 1GB RAM, P4 3GHz
>
> My stored procedure call went from 47ms avg to 6 ms average execution
> time on server, as reported by turning logging on
> All queries in general appeared to run between 5x to 10x faster.
>
>
>
>
>
--
=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/
If you must use inserts, you'll get much better performance if you wrap the restore in a transaction. I think someone came up with a clever way to do this on the command line, or you could edit the dump file. Note that if you do this and there's any errors, the restore will fail. Also, greatly increasing maintenance_work_mem during a restore can help with the index builds. On Aug 3, 2006, at 11:05 PM, Benjamin Krajmalnik wrote: > I believe I have to use onserts due to the problem with the records > with the improper utf encoding. According to the docs this will > cause the restore to fail. > > I will try it as well as a file system restore. > > -----Original Message----- > From: Joshua D. Drake <jd@commandprompt.com> > To: Benjamin Krajmalnik <kraj@illumen.com> > CC: pgsql-admin@postgresql.org <pgsql-admin@postgresql.org> > Sent: Thu Aug 03 21:01:54 2006 > Subject: Re: [ADMIN] File system backup question > > Benjamin Krajmalnik wrote: > > I am currently running PG 8.1.4 windows. > > The system is for a real time monitoring application (so downtime > needs > > to be minimized if at all possible). > > > > I will be migratng to a new server running PG 8.1.4 on FreeBSD 6.1. > > I have been playing with various options for migrating the data. > The > > database is currently about 8.5 GB. > > pg_dump took about 90 minutes. > > pg_restore has been running for over an hour and is abou 4% done > (based > > on row counts on the tables). > > > > I used the pg_dump option to insert records (as oposed to use the > copy > > command - I don't know which would be faster for the restore). > > That is why it is taking so long. > > Do not use inserts > Turn off fsync > Crank up shared buffers > make sure you have lots of checkpoint_segments. > > Joshua D. Drake > > > > > > In any case, I was thinking about performing a file system backup. > > My concern is that I am movinf between different OS's (although > both are > > i386 architecture). I will probably try it tomorrow to see if it > works, > > but was just wondering if there are any caveats which I should be > aware > > of. > > I hope this works, since for my scenario, this will have the least > > downtime. > > > > Thanks in advance for any advice. > > > > Regards, > > > > Benjamin > > > > > > > > PS. For those who have asked about performance, Windows is much > slower > > than FreeBSD. > > > > Machine specs are as follows: > > > > Windows 2003 server, HP DL360, SCSI RAID-1, 2GB RAM, 2.8GHz XEON HT > > FreeBSD 6.1, SuperMicro 5014C-T, SATA RAID-1, 1GB RAM, P4 3GHz > > > > My stored procedure call went from 47ms avg to 6 ms average > execution > > time on server, as reported by turning logging on > > All queries in general appeared to run between 5x to 10x faster. > > > > > > > > > > > > > -- > > === The PostgreSQL Company: Command Prompt, Inc. === > Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 > Providing the most comprehensive PostgreSQL solutions since 1997 > http://www.commandprompt.com/ > > > -- Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com Pervasive Software http://pervasive.com work: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461