Обсуждение: Re: [PERFORM] 7.3.2 pg_restore very slow

Saranya Sivakumar wrote:
> Hi All,
>
> I am trying to back up a full copy of one of our databases (14G) and
> restore it on another server. Both databases run 7.3.2 version.
> Though the restore completed successfully, it took 9 hours for the
> process to complete. The destination server runs Fedora Core 3 with
> 512 MB RAM and has 1 processor. I have also deferred referential
> intergrity checks during the restore. I tried to tune some parameters
> in the config file, but it still takes 9 hours.

Firstly, you should upgrade to the most recent version of 7.3.x (7.3.15)
- that's a *lot* of bug-fixes you are missing

Then, I would temporarily disable fsync and increase sort_mem and
checkpoint_segments. What you're trying to do is make a single process
run as fast as possible, so allow it to grab more resources than you
normally would.

--
Richard Huxton
Archonet Ltd

>   IpcMemoryCreate: shmget(key=5432001, size=85450752, 03600) failed: Invalid argument
>   This error usually means that PostgreSQL's request for a shared memory
> segment exceeded your kernel's SHMMAX parameter.  You can either
> reduce the request size or reconfigure the kernel with larger SHMMAX.
> To reduce the request size (currently 85450752 bytes), reduce
> PostgreSQL's shared_buffers parameter (currently 10000) and/or
> its max_connections parameter (currently 128).
>   If the request size is already small, it's possible that it is less than
> your kernel's SHMMIN parameter, in which case raising the request size or
> reconfiguring SHMMIN is called for.

if you cat /proc/sys/kernel/shmmax
it will tell you what it is set to. It needs to be at least "85450752". The size that Postgresql
is trying to grab.

also shmall may need to be adjusted also.

>   The total RAM available on this machine is 512MB.
>
>   I am not sure how to set these parameters SHMMAX and SHMMIN.
>   Any help/advice would be greatly appreciated.

http://www.postgresql.org/docs/8.1/interactive/kernel-resources.html
This will help you to set the kernel parameters.

Regards,

Richard Broersma Jr.

> IpcMemoryCreate: shmget(key=5432001, size=85450752, 03600) failed: Invalid argument
> This error usually means that PostgreSQL's request for a shared memory
> segment exceeded your kernel's SHMMAX parameter. You can either
> reduce the request size or reconfigure the kernel with larger SHMMAX.
> To reduce the request size (currently 85450752 bytes), reduce
> PostgreSQL's shared_buffers parameter (currently 10000) and/or
> its max_connections parameter (currently 128).
> If the request size is already small, it's possible that it is less than
> your kernel's SHMMIN parameter, in which case raising the request size or
> reconfiguring SHMMIN is called for.

if you cat /proc/sys/kernel/shmmax
it will tell you what it is set to. It needs to be at least "85450752". The size that Postgresql
is trying to grab.

also shmall may need to be adjusted also.

> The total RAM available on this machine is 512MB.
>
> I am not sure how to set these parameters SHMMAX and SHMMIN.
> Any help/advice would be greatly appreciated.

http://www.postgresql.org/docs/8.1/interactive/kernel-resources.html
This will help you to set the kernel parameters.

Regards,

Richard Broersma Jr.

>   Thank you very much for the information. The SHMMAX was set to 33554432, and that's why it
> failed to start the postmaster. Thanks for the link to the kernel resources article. I guess
> changing these parameters would require recompiling the kernel.
>
>   Is there any work around without changing these parameters to make maximum use of RAM?
>
>   We got a new server now with 2GB RAM, but it also has the same value for SHMMAX.
>
>   And I am trying the restore with the following conf
>   sort_mem = 40960  (changed from 1024)
>   shared_buffers = 3000 (changed from 64)
>     max_connections = 128 (changed from 32)

This is one of the best links that I can give you in addition to the Postgresql Kernel resource
link.
http://www.powerpostgresql.com/PerfList

I am pretty much a beginner at resource tuning also.  In fact, after googling for sources that
describe how to tune kernel parameters, the postgresql documents remains the best documents I've
found so far.

I would be interested if anyone else on the list knows of any resources or books that have an in
depth discussion on methods/strategies to tune kernel parameters to maximized usage of system
resources and at the same time allow for harmonious sharing between various programs/services.

Regards,

Richard Broersma Jr.

> Thank you very much for the information. The SHMMAX was set to 33554432, and that's why it
> failed to start the postmaster. Thanks for the link to the kernel resources article. I guess
> changing these parameters would require recompiling the kernel.
>
> Is there any work around without changing these parameters to make maximum use of RAM?
>
> We got a new server now with 2GB RAM, but it also has the same value for SHMMAX.
>
> And I am trying the restore with the following conf
> sort_mem = 40960 (changed from 1024)
> shared_buffers = 3000 (changed from 64)
> max_connections = 128 (changed from 32)

This is one of the best links that I can give you in addition to the Postgresql Kernel resource
link.
http://www.powerpostgresql.com/PerfList

I am pretty much a beginner at resource tuning also. In fact, after googling for sources that
describe how to tune kernel parameters, the postgresql documents remains the best documents I've
found so far.

I would be interested if anyone else on the list knows of any resources or books that have an in
depth discussion on methods/strategies to tune kernel parameters to maximized usage of system
resources and at the same time allow for harmonious sharing between various programs/services.

Regards,

Richard Broersma Jr.

Hi, Saranya,

Saranya Sivakumar wrote:
> Thank you very much for the information. The SHMMAX was set to 33554432,
> and that's why it failed to start the postmaster. Thanks for the link to
> the kernel resources article. I guess changing these parameters would
> require recompiling the kernel.

As stated on the
http://www.postgresql.org/docs/8.1/interactive/kernel-resources.html
page, those values can be changed via sysctl or echoing values into
/proc, under linux at least.

HTH,
Markus

--
Markus Schaber | Logical Tracking&Tracing International AG
Dipl. Inf.     | Software Development GIS

Fight against software patents in EU! www.ffii.org www.nosoftwarepatents.org