Обсуждение: The right SHMMAX and FILE_MAX

Hi. I'm on a 64 Bit CentOS 5 system, quadcore processor, 8GB RAM and
tons of data storage (1 TB SATAII disks).

The current SHMMAX and SHMMIN are (commas added for legibility) --

kernel.shmmax = 68,719,476,736
kernel.shmall = 4,294,967,296

Now, according to my reading in the PG manual and this list, a good
recommended value for SHMMAX is

   (shared_buffers * 8192)

My postgresql.conf settings at the moment are:

    max_connections = 300
    shared_buffers = 300MB
    effective_cache_size = 2000MB

By this calculation, shared_b * 8192 will be:

     2,457,600,000,000

That's a humongous number. So either the principle for SHMMAX is
amiss, or I am reading this wrongly?

Similarly with "fs.file_max". There are articles like this one:
http://tldp.org/LDP/solrhe/Securing-Optimizing-Linux-RH-Edition-v1.3/chap6sec72.html

Is this relevant for PostgreSQL performance at all, or should I skip that?

Thanks for any pointers!

Phoenix Kiula <phoenix.kiula@gmail.com> wrote:

> Now, according to my reading in the PG manual and this list, a
> good recommended value for SHMMAX is
>
>    (shared_buffers * 8192)

Where did you see that?  The amount of data buffered is the number
of shared buffers * 8KB.  Taking shared_buffers as a number of bytes
and multiplying by 8K makes no sense at all.  Any documentation
which can be read to indicate that should be fixed.

Besides that, there is shared memory space needed besides the actual
buffered disk pages, so you're not looking at the whole picture once
you stop dealing with "bytes squared".

-Kevin

On 05/01/2011 02:48 AM, Phoenix Kiula wrote:
> Hi. I'm on a 64 Bit CentOS 5 system, quadcore processor, 8GB RAM and
> tons of data storage (1 TB SATAII disks).
>
> The current SHMMAX and SHMMIN are (commas added for legibility) --
>
> kernel.shmmax = 68,719,476,736
> kernel.shmall = 4,294,967,296
>

That's set higher than the amount of RAM in the server.  Run the
attached script; it will produce reasonable values for your server,
presuming you'll never want to allocate >50% of the RAM in the server
for shared memory.  Given standard tuning for shared_buffers is <40%,
I've never run into a situation where this was a terrible choice if you
want to just set and forget about it.  Only reason to fine-tine is if
another major user of shared memory is running on the server

> Now, according to my reading in the PG manual and this list, a good
> recommended value for SHMMAX is
>
>     (shared_buffers * 8192)
>

The value for shared_buffers stored internally is in 8192 byte pages:

select setting,unit,current_setting(name) from pg_settings where
name='shared_buffers';
  setting | unit | current_setting
---------+------+-----------------
  4096    | 8kB  | 32MB

So any formula you found that does this sort of thing is just converting
it back to bytes again, and is probably from an earlier PostgreSQL
version where you couldn't set this parameter in memory units.  SHMMAX
needs to be a bit bigger than shared_buffers in bytes.

> Similarly with "fs.file_max". There are articles like this one:
> http://tldp.org/LDP/solrhe/Securing-Optimizing-Linux-RH-Edition-v1.3/chap6sec72.html
> Is this relevant for PostgreSQL performance at all, or should I skip that?
>

That's ancient history.  This is how big the default is on the two Linux
distributions I have handy:

[RHEL5]
$ cat /proc/sys/fs/file-max
745312

[Debian Squeeze]
$ cat /proc/sys/fs/file-max
1645719

It was a tiny number circa the RedHat 6 that manual was written for, now
it's very unlikely you'll exceed the kernel setting here.

--
Greg Smith   2ndQuadrant US    greg@2ndQuadrant.com   Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support  www.2ndQuadrant.us
"PostgreSQL 9.0 High Performance": http://www.2ndQuadrant.com/books

I am also in need of a proper documentation that explains how to set
SHMAX and SHMALL variables in Postgres.

What things need to be taken in consideration before doing that ?
What is the value of SHMAX & SHMALL if u have 16 GB RAM for Postgres
Server ?



Thanks

Phoenix Kiula wrote:
> Hi. I'm on a 64 Bit CentOS 5 system, quadcore processor, 8GB RAM and
> tons of data storage (1 TB SATAII disks).
>
> The current SHMMAX and SHMMIN are (commas added for legibility) --
>
> kernel.shmmax = 68,719,476,736
> kernel.shmall = 4,294,967,296
>
> Now, according to my reading in the PG manual and this list, a good
> recommended value for SHMMAX is
>
>    (shared_buffers * 8192)
>
> My postgresql.conf settings at the moment are:
>
>     max_connections = 300
>     shared_buffers = 300MB
>     effective_cache_size = 2000MB
>
> By this calculation, shared_b * 8192 will be:
>
>      2,457,600,000,000
>
> That's a humongous number. So either the principle for SHMMAX is
> amiss, or I am reading this wrongly?
>
> Similarly with "fs.file_max". There are articles like this one:
> http://tldp.org/LDP/solrhe/Securing-Optimizing-Linux-RH-Edition-v1.3/chap6sec72.html
>
> Is this relevant for PostgreSQL performance at all, or should I skip that?
>
> Thanks for any pointers!
>
>

On 01/05/11 18:48, Phoenix Kiula wrote:
> Now, according to my reading in the PG manual and this list, a good
> recommended value for SHMMAX is
>
>     (shared_buffers * 8192)
>
> My postgresql.conf settings at the moment are:
>
>      max_connections = 300
>      shared_buffers = 300MB
>      effective_cache_size = 2000MB
>
> By this calculation, shared_b * 8192 will be:
>
>       2,457,600,000,000
>
> That's a humongous number. So either the principle for SHMMAX is
> amiss, or I am reading this wrongly?

You are confusing shared_buffers expressed as "number of pages" with
shared_buffers expressed as "MB". The docs are assuming you are working
with the former (and would appear to be assuming your pagesize is 8K -
which is teh default but not required to be the case). If you are
showing shared_buffers as "MB" then obviously you cane set SHMMAX using
the value multiplied by (1024*1024), so in your case:

300 * (1024*1024) = 314572800


However note that there are things other than shared_buffers require
shared memory, so you'll need more than this. Use Greg's script.

Cheers

Mark

Dne 2.5.2011 06:53, Adarsh Sharma napsal(a):
> I am also in need of a proper documentation that explains how to set
> SHMAX and SHMALL variables in Postgres.
>
> What things need to be taken in consideration before doing that ?
> What is the value of SHMAX & SHMALL if u have 16 GB RAM for Postgres
> Server ?

Well, those two values actually define kernel limits for shared memory
segments (i.e. memory shared by multiple processes, in this case the
postmaster proces and backends). So it's rather a question of tuning
shared_buffers (because that's the shared memory segment) and then
setting those two values.

SHMMAX - max. size of a single shared segment (in bytes)
SHMALL - total size of shared segments (in pages, page is usually 4kB)

So if you decide you want 1GB shared buffers, you'll need at least this

SHMMAX = 1024 * 1024 * 1024 (i.e. 1GB)
SHMALL = 1024 * 256 (1GB in 4kB pages)

(althouth the SHMALL should be higher, as there will be other processes
that need shared memory).

There's a lot of docs about this, e.g. this one (it's mostly for Oracle,
but it describes the shared memory quite nicely):

http://www.puschitz.com/TuningLinuxForOracle.shtml#SettingSharedMemory

regards
Tomas

Dne 2.5.2011 06:53, Adarsh Sharma napsal(a): 

I am also in need of a proper documentation that explains how to set 
SHMAX and SHMALL variables in Postgres.

What things need to be taken in consideration before doing that ?
What is the value of SHMAX & SHMALL if u have 16 GB RAM for Postgres
Server ?   

Well, those two values actually define kernel limits for shared memory
segments (i.e. memory shared by multiple processes, in this case the
postmaster proces and backends). So it's rather a question of tuning
shared_buffers (because that's the shared memory segment) and then
setting those two values. 

SHMMAX - max. size of a single shared segment (in bytes)
SHMALL - total size of shared segments (in pages, page is usually 4kB)

So if you decide you want 1GB shared buffers, you'll need at least this

SHMMAX = 1024 * 1024 * 1024 (i.e. 1GB)
SHMALL = 1024 * 256 (1GB in 4kB pages)

(althouth the SHMALL should be higher, as there will be other processes
that need shared memory).

There's a lot of docs about this, e.g. this one (it's mostly for Oracle,
but it describes the shared memory quite nicely):

http://www.puschitz.com/TuningLinuxForOracle.shtml#SettingSharedMemory

regards
Tomas
 

On 05/02/2011 12:53 AM, Adarsh Sharma wrote:
> I am also in need of a proper documentation that explains how to set
> SHMAX and SHMALL variables in Postgres.
>
> What things need to be taken in consideration before doing that ?
> What is the value of SHMAX & SHMALL if u have 16 GB RAM for Postgres
> Server ?

Running the script I provided on a Linux server with 16GB of RAM:

$ cat /proc/meminfo | grep MemTotal
MemTotal:       16467464 kB

$ getconf PAGE_SIZE
4096

$ ./shmsetup
# Maximum shared segment size in bytes
kernel.shmmax = 8431341568
# Maximum number of shared memory segments in pages
kernel.shmall = 2058433

That sets SHMMAX to 50% of total RAM, which should be sufficient for any
reasonable installation of PostgreSQL.  shmmall is a similar limit
expressed in system pages instead, so as seen here you have to divide by
the system page size of 4096 to figure it out.

--
Greg Smith   2ndQuadrant US    greg@2ndQuadrant.com   Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support  www.2ndQuadrant.us
"PostgreSQL 9.0 High Performance": http://www.2ndQuadrant.com/books