shared memory settings on MAC OS X

Поиск
Список
Период
Сортировка
Искать

shared memory settings on MAC OS X

От:
Maximilian Tyrtania <mty@fischerappelt.de>
Дата:
Hi there,

I am trying to increase the amount of shared memory on my Mac OS 10.4
machine (i have 5 GB RAM installed), but without too much success.

My /etc/sysctl.conf file looks like this:
                
kern.sysv.shmmax=33554432
kern.sysv.shmmin=1
kern.sysv.shmmni=32
kern.sysv.shmseg=8
kern.sysv.shmall=8192

Then I reboot and do sysctl -a | grep shm, and read:

kern.sysv.shmmax=4194304
kern.sysv.shmmin=1
kern.sysv.shmmni=32
kern.sysv.shmseg=8
kern.sysv.shmall=8192


4194304 is the default value for kern.sysv.shmmax.
It looks as if Tiger ignores my feeble attempt to set the kern.sysv.shmmax
value.

Any pointers? 

BTW, as I intend to allow 240 connections I should probably raise the
settings even way beyond that, I am just still experimenting.

Best,

- Maximilian Tyrtania | mty@fischerappelt.de
  fischerAppelt Kommunikation GmbH
  Tucholskystr.18 | D-10117 Berlin | Germany
  Tel. ++49-30-726146-728
  http://www.fischerappelt.de

--


Re: shared memory settings on MAC OS X

От:
Tom Lane <tgl@sss.pgh.pa.us>
Дата:
Maximilian Tyrtania  writes:
> I am trying to increase the amount of shared memory on my Mac OS 10.4
> machine (i have 5 GB RAM installed), but without too much success.

Weird, those same exact settings work fine for me.  How up-to-date
is your 10.4.x installation?  The relevant part of /etc/rc in mine is


if [ -f /etc/sysctl-macosxserver.conf ]; then
        awk '{ if (!index($1, "#") && index($1, "=")) print $1 }' < /etc/sysctl\
-macosxserver.conf | while read
        do
                sysctl -w ${REPLY}
        done
fi
if [ -f /etc/sysctl.conf ]; then
        awk '{ if (!index($1, "#") && index($1, "=")) print $1 }' < /etc/sysctl\
.conf | while read
        do
                sysctl -w ${REPLY}
        done
fi

sysctl -w kern.sysv.shmmax=4194304 kern.sysv.shmmin=1 kern.sysv.shmmni=32 kern.\
sysv.shmseg=8 kern.sysv.shmall=1024


So one obvious gotcha would be if you have a
/etc/sysctl-macosxserver.conf file that sets 'em differently.

The gotchas I know about in this area are:

* The OSX kernel seems to lock down the shmem settings as soon as all
five have been correctly specified via sysctl.  This is why the sysctl
done last in /etc/rc doesn't overwrite yours.

* It's fairly draconian about what "correctly specified" is --- notably,
at least some versions insist on shmmax being an exact multiple of the
page size.  But the numbers you quoted look OK.

			regards, tom lane

Re: shared memory settings on MAC OS X

От:
Maximilian Tyrtania <mty@fischerappelt.de>
Дата:
Hi,

>> I am trying to increase the amount of shared memory on my Mac OS 10.4
>> machine (i have 5 GB RAM installed), but without too much success.
> 
> Weird, those same exact settings work fine for me.  How up-to-date
> is your 10.4.x installation?

Ah, sorry, it's up to date, 10.4.10.

>  The relevant part of /etc/rc in mine is
> 
> 
> if [ -f /etc/sysctl-macosxserver.conf ]; then
>         awk '{ if (!index($1, "#") && index($1, "=")) print $1 }' <
> /etc/sysctl\
> -macosxserver.conf | while read
>         do
>                 sysctl -w ${REPLY}
>         done
> fi
> if [ -f /etc/sysctl.conf ]; then
>         awk '{ if (!index($1, "#") && index($1, "=")) print $1 }' <
> /etc/sysctl\
> .conf | while read
>         do
>                 sysctl -w ${REPLY}
>         done
> fi
> 
> sysctl -w kern.sysv.shmmax=4194304 kern.sysv.shmmin=1 kern.sysv.shmmni=32
> kern.\
> sysv.shmseg=8 kern.sysv.shmall=1024

Thanks for digging that out.

> So one obvious gotcha would be if you have a
> /etc/sysctl-macosxserver.conf file that sets 'em differently.

Good to know. Unfortunately I don't seem to have that file. Weird.

> The gotchas I know about in this area are:
> 
> * The OSX kernel seems to lock down the shmem settings as soon as all
> five have been correctly specified via sysctl.  This is why the sysctl
> done last in /etc/rc doesn't overwrite yours.

> * It's fairly draconian about what "correctly specified" is --- notably,
> at least some versions insist on shmmax being an exact multiple of the
> page size.  But the numbers you quoted look OK.

I tried other numbers as well. Tiger wouldn't change that shmmax value from
4194304. Not sure what's going on there.

Thanks again,

- Maximilian Tyrtania | mty@fischerappelt.de
  fischerAppelt Kommunikation GmbH
  Tucholskystr.18 | D-10117 Berlin | Germany
  Tel. ++49-30-726146-728
  http://www.fischerappelt.de

--


Re: shared memory settings on MAC OS X

От:
Tom Lane <tgl@sss.pgh.pa.us>
Дата:
Maximilian Tyrtania  writes:
> I tried other numbers as well. Tiger wouldn't change that shmmax value from
> 4194304. Not sure what's going on there.

Weird.  As a debugging measure, see what happens if you change that last
sysctl line in /etc/rc.

			regards, tom lane

Re: shared memory settings on MAC OS X

От:
Maximilian Tyrtania <mty@fischerappelt.de>
Дата:
Hi again,

Tom, I suspect you have Mac OS X Server installed, right? That's probably
why your /etc/sysctl.conf file mentions that /etc/sysctl-macosxserver.conf
file, while mine doesn't.

The critical part in my /etc/sysctl.conf file looks like this:

if [ -f /etc/sysctl.conf ]; then
        awk '{ if (!index($1, "#") && index($1, "=")) print $1 }' <
/etc/sysctl.conf | while read
        do
                sysctl -w ${REPLY}
        done
fi

sysctl -w kern.sysv.shmmax=4194304 kern.sysv.shmmin=1 kern.sysv.shmmni=32
kern.sysv.shmseg=8 kern.sysv.shmall=1024


And frankly, to me it looks as if it means "if there is a /etc/sysctl.conf
file, then read it and accept its settings. Then overwrite the sysctl
settings with the default values, no matter what."

I know that my understanding is wrong though, because I tried changing the
settings on my local machine via the /etc/sysctl.conf file, and it worked.
But not on the machine that's supposed to be our pg server.

> As a debugging measure, see what happens if you change that last
> sysctl line in /etc/rc.

Okay, I will try that.

Thanks,

  Maximilian Tyrtania | IT-Entwickler
  Mty@fischerAppelt.de
  fischerAppelt Kommunikation GmbH
  Friedichstr. 149 | D-10117 Berlin | Germany
  Tel. ++49-30-726146-728 | Fax ++49-30-726146-710
  www.fischerAppelt.de
---------------------------------------------------------------------------
Sitz der Gesellschaft: Hamburg
Standorte: Hamburg, Berlin, Düsseldorf, Frankfurt, München, Stuttgart
Die Gesellschaft ist Teil des Everpublic Networks
Handelsregister: Amtsgericht Hamburg, HRB 49259
Geschäftsführer: Andreas Fischer-Appelt, Bernhard Fischer-Appelt

+++ fischerAppelt macht Fernsehen: www.fischerappelt-tvmedia.com +++




Re: shared memory settings on MAC OS X

От:
Alexander Stanier <alexander.stanier@egsgroup.com>
Дата:
 We edit the /etc/rc directly; comment out the line with the default settings and add a new one just underneath it. The only time this can cause a problem is that softwareupdate has been known to overwrite our settings from time to time, but you just have to check after patching and put it back in if goes. Probably not the right way to do it, but it has always worked for us!
Alex Stanier
EGS Limited

Direct Line: +44 (0) 20 7539 2811
Switchboard: +44 (0) 20 7539 2828
Fax: +44 (0) 20 7539 2829
www.egsgroup.com

Registered Address: Baird House, 15-17 St Cross Street, London EC1N 8UW
Registered Number: 4006942


Maximilian Tyrtania wrote:
Hi again,

Tom, I suspect you have Mac OS X Server installed, right? That's probably
why your /etc/sysctl.conf file mentions that /etc/sysctl-macosxserver.conf
file, while mine doesn't.

The critical part in my /etc/sysctl.conf file looks like this:

if [ -f /etc/sysctl.conf ]; then       awk '{ if (!index($1, "#") && index($1, "=")) print $1 }' <
/etc/sysctl.conf | while read       do               sysctl -w ${REPLY}       done
fi

sysctl -w kern.sysv.shmmax=4194304 kern.sysv.shmmin=1 kern.sysv.shmmni=32
kern.sysv.shmseg=8 kern.sysv.shmall=1024


And frankly, to me it looks as if it means "if there is a /etc/sysctl.conf
file, then read it and accept its settings. Then overwrite the sysctl
settings with the default values, no matter what."

I know that my understanding is wrong though, because I tried changing the
settings on my local machine via the /etc/sysctl.conf file, and it worked.
But not on the machine that's supposed to be our pg server.
 
As a debugging measure, see what happens if you change that last
sysctl line in /etc/rc.   
Okay, I will try that.

Thanks,
 Maximilian Tyrtania | IT-Entwickler Mty@fischerAppelt.de fischerAppelt Kommunikation GmbH Friedichstr. 149 | D-10117 Berlin | Germany Tel. ++49-30-726146-728 | Fax ++49-30-726146-710 www.fischerAppelt.de
---------------------------------------------------------------------------
Sitz der Gesellschaft: Hamburg
Standorte: Hamburg, Berlin, Düsseldorf, Frankfurt, München, Stuttgart
Die Gesellschaft ist Teil des Everpublic Networks
Handelsregister: Amtsgericht Hamburg, HRB 49259
Geschäftsführer: Andreas Fischer-Appelt, Bernhard Fischer-Appelt

+++ fischerAppelt macht Fernsehen: www.fischerappelt-tvmedia.com +++





---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to      choose an index scan if your joining column's datatypes do not      match


This email has been scanned for viruses by Blackspider's Mail Control System. 
For more information please visit http://www.blackspider.com  
 


This message has been scanned for malware by SurfControl plc. www.surfcontrol.com

Re: shared memory settings on MAC OS X

От:
Tom Lane <tgl@sss.pgh.pa.us>
Дата:
Maximilian Tyrtania  writes:
> Tom, I suspect you have Mac OS X Server installed, right? That's probably
> why your /etc/sysctl.conf file mentions that /etc/sysctl-macosxserver.conf
> file, while mine doesn't.

Uh, no, I'm looking at my laptop.  Curious that yours has no reference
to the other file.

> And frankly, to me it looks as if it means "if there is a /etc/sysctl.conf
> file, then read it and accept its settings. Then overwrite the sysctl
> settings with the default values, no matter what."

You're forgetting the point I made that the first complete set of shmem
settings wins.  If we could change the settings on the fly after that,
all this would be a whole lot easier, but the OSX kernel locks them down
somehow.

BTW, I dunno if you read awk at all, but that awk command effectively
says "print lines that contain = and do not contain #".  You didn't try
appending comments to the setting lines in your file did you?

			regards, tom lane

Re: shared memory settings on MAC OS X

От:
Maximilian Tyrtania <mty@fischerappelt.de>
Дата:
> Von: Tom Lane 
> Datum: Mon, 29 Oct 2007 09:40:10 -0400

>> Tom, I suspect you have Mac OS X Server installed, right? That's probably
>> why your /etc/sysctl.conf file mentions that /etc/sysctl-macosxserver.conf
>> file, while mine doesn't.
> 
> Uh, no, I'm looking at my laptop.  Curious that yours has no reference
> to the other file.

Indeed. Good to hear you are using a Mac, though.

>> And frankly, to me it looks as if it means "if there is a /etc/sysctl.conf
>> file, then read it and accept its settings. Then overwrite the sysctl
>> settings with the default values, no matter what."
> 
> You're forgetting the point I made that the first complete set of shmem
> settings wins. 

Of course I only understood what you said there after I hit the send button.
Now it clicked, thanks for putting it so politely.

> If we could change the settings on the fly after that,
> all this would be a whole lot easier, but the OSX kernel locks them down
> somehow.

Yeah, I wonder if this locking down somehow doesn't work on my machine. I
mean I successfully changed all of the other shm value via that file, only
that shmmax value would'nt budge.

> BTW, I dunno if you read awk at all, but that awk command effectively
> says "print lines that contain = and do not contain #".

Ah, no I wasn't aware of this.

> You didn't try
> appending comments to the setting lines in your file did you?

No, i didn't. 

I think I will try messing with the rc file directly.

Thanks,

  Maximilian Tyrtania | IT-Entwickler
  Mty@fischerAppelt.de
  fischerAppelt Kommunikation GmbH
  Friedichstr. 149 | D-10117 Berlin | Germany
  Tel. ++49-30-726146-728 | Fax ++49-30-726146-710
  www.fischerAppelt.de
---------------------------------------------------------------------------
Sitz der Gesellschaft: Hamburg
Standorte: Hamburg, Berlin, Düsseldorf, Frankfurt, München, Stuttgart
Die Gesellschaft ist Teil des Everpublic Networks
Handelsregister: Amtsgericht Hamburg, HRB 49259
Geschäftsführer: Andreas Fischer-Appelt, Bernhard Fischer-Appelt

+++ fischerAppelt macht Fernsehen: www.fischerappelt-tvmedia.com +++




Re: shared memory settings on MAC OS X

От:
Brendan Duddridge <brendan@clickspace.com>
Дата:
Вложения:
Either way, when you do manage to change your shared memory settings,  
you still won't be able to use more than about 4 GB of shared buffer  
space in Postgres, or any other program for that matter. We thought  
Leopard would help with this, but it hasn't. We have a 32 GB dual 3  
Ghz Intel Xeon Xserve, but we can't set our shared buffers as high as  
we'd like. We also have a similarly spec'd Linux machine and it has no  
problem allowing us to specify lots of shared buffers. For some  
reason, OS X gives an error when you try to attach to anything more  
than 4 GB.

Here's a program we wrote to demonstrate the problem, which we have  
submitted to Apple as a bug (Radar #5547628):

/*** create_shared_memory.c ***/
/* Takes as input from standard in a size in bytes for a shared memory  
segment
that this program will attempt to create and attach to. If any errors  
occur
trying to create or attach to the shared memory, they are printed to  
standard
out. This program will then attempt to detach from and delete the  
shared memory
it allocated and/or attached to as appropriate.

On Mac OS X 10.5 Server (9A557) on an Intel Quad core Xserve,  
compiling this
program with gcc version i686-apple-darwin9-gcc-4.0.1 as follows
#gcc -arch x86_64 create_shared_memory.c
you will see the maximum shared memory segment that can be attached to  
is
4292870144. I do not know if this shared memory limit applies to the  
final
version of Leopard Server or on other hardware.

Note: You will need greater than 4GB of memory on your machine and  
large enough
values set for kern.sysv.shmall and kern.sysv.shmmax to verify this  
shared
memory limit.

Known limitations: This program does not provide any output if the  
allocated
shared memory cannot be deallocated or if the attached shared memory  
cannot be
detached.

Written by: Brian Wipf
ClickSpace Interactive Inc.
Date: October 18, 2007
*/

#include 
#include 
#include 
#include 
#include 

typedef struct {
    int iPID;
} shared_t;

main()
{
    int key = 912367102; /* Some random key for the shared memory  
segment. */
    size_t size;
    int shmflg=IPC_CREAT | 0600;
    int shmid;
    shared_t  *shared_mem;

/* If you want to specify your own key, uncomment the next two lines  
of code. */
/*
    printf("Enter an integer key for shared memory segment: ");
    scanf("%d", &key);
*/
    printf("Enter size for shared memory segment (in bytes): ");
    scanf("%ld", &size);

    /* Create the shared memory segment */
    printf("\nAttempting to create the shared memory segment...\n");
    shmid=shmget(key, size, shmflg);

    if (errno != 0) {
       printf("There was an error trying to allocate the requested  
amount of shared memory\n");
       printf("errno: %d\n", errno);
    } else {
       printf("Shared memory created successfully\n");

       /* Attach to the shared memory segment */
       printf("\nAttempting to attach to the shared memory segment... 
\n");
       shared_mem = (shared_t *) shmat ( shmid, 0, 0);

       if (errno != 0) {
          printf("Error trying to attach to the shared memory segment. 
\n");
          printf("errno: %d\n\n", errno);
       } else {
          printf("Shared memory attached to successfully\n");

          /* Detach from shared memory segment */
          printf("\nDetaching from the shared memory segment\n\n");
          shmdt(shared_mem);
       }

       /* Delete the shared memory segment */
       printf("Deleting the shared memory segment\n");
       shmctl(shmid, IPC_RMID, NULL);
    }

    return 0;
}




____________________________________________________________________
Brendan Duddridge | CTO | 403-277-5591 x24 |  brendan@clickspace.com

ClickSpace Interactive Inc.
Suite L100, 239 - 10th Ave. SE
Calgary, AB  T2G 0V9

http://www.clickspace.com

On Oct 29, 2007, at 7:40 AM, Tom Lane wrote:

> Maximilian Tyrtania  writes:
>> Tom, I suspect you have Mac OS X Server installed, right? That's  
>> probably
>> why your /etc/sysctl.conf file mentions that /etc/sysctl- 
>> macosxserver.conf
>> file, while mine doesn't.
>
> Uh, no, I'm looking at my laptop.  Curious that yours has no reference
> to the other file.
>
>> And frankly, to me it looks as if it means "if there is a /etc/ 
>> sysctl.conf
>> file, then read it and accept its settings. Then overwrite the sysctl
>> settings with the default values, no matter what."
>
> You're forgetting the point I made that the first complete set of  
> shmem
> settings wins.  If we could change the settings on the fly after that,
> all this would be a whole lot easier, but the OSX kernel locks them  
> down
> somehow.
>
> BTW, I dunno if you read awk at all, but that awk command effectively
> says "print lines that contain = and do not contain #".  You didn't  
> try
> appending comments to the setting lines in your file did you?
>
> 			regards, tom lane
>
> ---------------------------(end of  
> broadcast)---------------------------
> TIP 2: Don't 'kill -9' the postmaster
>

FAQ