Обсуждение: Max backend limits cleaned up
I have just checked in code changes (no doc updates yet :-() that
address our recent discussions about how many backend processes
can be used. Specifically:
configure takes a --with-maxbackends=N switch that sets the hard limit
on the maximum number of backends per postmaster. (It's still a hard
limit because several arrays are sized by MAXBACKENDS. I didn't think
it was worth trying to change that.) The default is still 64.
The postmaster can be started with a "-N backends" switch that sets
a smaller limit on the number of backends for this postmaster.
The only cost of having a large MAXBACKENDS constant is a few dozen
bytes of shared memory per array slot, so if you want you can configure
MAXBACKENDS pretty large and then set the effective limit with -N at
postmaster startup.
When the postmaster is started, it will immediately acquire enough
semaphores to support min(MAXBACKENDS, -N) backend processes.
If your kernel sema parameters are too low to allow that, you get an
immediate failure, rather than failure under peak load. The postmaster
just refuses to start up, with a log message like this:IpcSemaphoreCreate: semget failed (No space left on device)
key=5440026,num=16, permission=600
(Right at this instant, it looks like it fails to release whatever
semas it did acquire. Ugh. Think I can fix that though.)
I have verified that I can start more than 64 backends after suitable
configuration, but I am not in a position to check that things work
smoothly with a really large number of backends. I found one parameter
(MAX_PROC_SEMS) that was hard-wired at 128 rather than set equal to
MaxBackendIds, so I am a little worried that there might be others.
If anyone has the time and interest to push the envelope with a few
hundred backends, please report back!
regards, tom lane
I did a little more work on the configurable-max-backends patch:
1. initdb didn't work because I had broken bootstrap mode :-(.
Fixed.
2. I separated the hard maximum limit on the number of backends
(MAXBACKENDS, used to size a couple of arrays) from the default
soft limit (now DEF_MAXBACKENDS).
3. The only cost of enlarging MAXBACKENDS is about 32 bytes per
array slot, so I set MAXBACKENDS at 1024 while leaving the
default DEF_MAXBACKENDS at 64. (I more than bought back the 32K
by reducing the number of allocated spinlocks to what we were
actually using, anyway.)
4. Upshot: default limit on number of backends is 64, same as it
ever was, but you can set it as high as 1024 without a recompile.
Just start the postmaster with desired -N switch. (Of course you
might need to reconfigure your kernel first ;-).)
5. Allocation of semaphores and shared memory is now based on
-N switch value (default or specified) rather than the MAXBACKENDS
constant.
regards, tom lane
> 5. Allocation of semaphores and shared memory is now based on > -N switch value (default or specified) rather than the MAXBACKENDS > constant. sgml and man documenation updates, right? Or should I do it? -- Bruce Momjian | http://www.op.net/~candle maillist@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
Bruce Momjian <maillist@candle.pha.pa.us> writes:
>> 5. Allocation of semaphores and shared memory is now based on
>> -N switch value (default or specified) rather than the MAXBACKENDS
>> constant.
> sgml and man documenation updates, right? Or should I do it?
I put something into the docs last night in the places where configure
and postmaster switches are described.
I am thinking, though, that we also ought to have FAQ entries under
headings like:
I get "IpcSemaphoreCreate: semget failed (No space left on device)"when I try to start the postmaster
I get 'Sorry, too many clients already' when trying to connect
If you like, I'll try to write up a first cut at these.
regards, tom lane
> Bruce Momjian <maillist@candle.pha.pa.us> writes: > >> 5. Allocation of semaphores and shared memory is now based on > >> -N switch value (default or specified) rather than the MAXBACKENDS > >> constant. > > > sgml and man documenation updates, right? Or should I do it? > > I put something into the docs last night in the places where configure > and postmaster switches are described. > > I am thinking, though, that we also ought to have FAQ entries under > headings like: > > I get "IpcSemaphoreCreate: semget failed (No space left on device)" > when I try to start the postmaster > > I get 'Sorry, too many clients already' when trying to connect > > If you like, I'll try to write up a first cut at these. Sure. -- Bruce Momjian | http://www.op.net/~candle maillist@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
I got it working by adding a -N 32 to the postmaster startup. Looks like my site BSD/OS can't start 64 backends. Some of my configuration is wrong. Perhaps we need 32 as the default. > Bruce Momjian <maillist@candle.pha.pa.us> writes: > >> 5. Allocation of semaphores and shared memory is now based on > >> -N switch value (default or specified) rather than the MAXBACKENDS > >> constant. > > > sgml and man documenation updates, right? Or should I do it? > > I put something into the docs last night in the places where configure > and postmaster switches are described. > > I am thinking, though, that we also ought to have FAQ entries under > headings like: > > I get "IpcSemaphoreCreate: semget failed (No space left on device)" > when I try to start the postmaster > > I get 'Sorry, too many clients already' when trying to connect > > If you like, I'll try to write up a first cut at these. > > regards, tom lane > -- Bruce Momjian | http://www.op.net/~candle maillist@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
Yom, I am getting: (1) cat /u/pg/server.log IpcSemaphoreCreate: semget failed (No space left on device) key=5432017, num=16, permission=600 This is without any special switches. Do I need to do anything? > Bruce Momjian <maillist@candle.pha.pa.us> writes: > >> 5. Allocation of semaphores and shared memory is now based on > >> -N switch value (default or specified) rather than the MAXBACKENDS > >> constant. > > > sgml and man documenation updates, right? Or should I do it? > > I put something into the docs last night in the places where configure > and postmaster switches are described. > > I am thinking, though, that we also ought to have FAQ entries under > headings like: > > I get "IpcSemaphoreCreate: semget failed (No space left on device)" > when I try to start the postmaster > > I get 'Sorry, too many clients already' when trying to connect > > If you like, I'll try to write up a first cut at these. > > regards, tom lane > -- Bruce Momjian | http://www.op.net/~candle maillist@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
Bruce Momjian <maillist@candle.pha.pa.us> writes:
> I am getting:
> IpcSemaphoreCreate: semget failed (No space left on device) key=5432017,
> num=16, permission=600
> [ later ]
> I got it working by adding a -N 32 to the postmaster startup. Looks
> like my site BSD/OS can't start 64 backends. Some of my configuration
> is wrong. Perhaps we need 32 as the default.
Yeah, I was thinking about that myself. I left the default -N setting
at 64 on the theory that people who had gone to the trouble of making
sure they had proper kernel configurations should not get surprised by
v6.5 suddenly reducing the default number-of-backends limit.
On the other hand, we have reason to believe that a lot of systems are
not configured to allow Postgres to grab 64 semaphores, so if we don't
reduce the default -N value we will almost certainly see a lot of gripes
just like the above when people move to 6.5. (I think -N 32 would work
as a default on minimally-configured systems, but cannot prove it.)
I haven't got a real strong feeling either way. Opinions?
regards, tom lane
Having recently experienced a similar problem with semaphores and kernel size, I can say it is an issue. I feel that documentation will clear it up either way. Either you lower the default backend limit, and document how to raise it along with the associated kernel variables, or leave it alone and document the appropriate steps to tuning the kernel to accommodate it and how to lower it if you don't want to tune the kernel. DwD > -----Original Message----- > From: owner-pgsql-hackers@postgreSQL.org > [mailto:owner-pgsql-hackers@postgreSQL.org]On Behalf Of Tom Lane > Sent: Monday, February 22, 1999 10:10 AM > To: Bruce Momjian > Cc: pgsql-hackers@postgreSQL.org > Subject: Re: [HACKERS] Re: Max backend limits cleaned up > > > Bruce Momjian <maillist@candle.pha.pa.us> writes: > > I am getting: > > IpcSemaphoreCreate: semget failed (No space left on > device) key=5432017, > > num=16, permission=600 > > [ later ] > > I got it working by adding a -N 32 to the postmaster > startup. Looks > > like my site BSD/OS can't start 64 backends. Some of > my configuration > > is wrong. Perhaps we need 32 as the default. > > Yeah, I was thinking about that myself. I left the > default -N setting > at 64 on the theory that people who had gone to the > trouble of making > sure they had proper kernel configurations should not get > surprised by > v6.5 suddenly reducing the default number-of-backends limit. > > On the other hand, we have reason to believe that a lot > of systems are > not configured to allow Postgres to grab 64 semaphores, > so if we don't > reduce the default -N value we will almost certainly see > a lot of gripes > just like the above when people move to 6.5. (I think -N > 32 would work > as a default on minimally-configured systems, but cannot > prove it.) > > I haven't got a real strong feeling either way. Opinions? > > regards, tom lane >
> Bruce Momjian <maillist@candle.pha.pa.us> writes: > > I am getting: > > IpcSemaphoreCreate: semget failed (No space left on device) key=5432017, > > num=16, permission=600 > > [ later ] > > I got it working by adding a -N 32 to the postmaster startup. Looks > > like my site BSD/OS can't start 64 backends. Some of my configuration > > is wrong. Perhaps we need 32 as the default. > > Yeah, I was thinking about that myself. I left the default -N setting > at 64 on the theory that people who had gone to the trouble of making > sure they had proper kernel configurations should not get surprised by > v6.5 suddenly reducing the default number-of-backends limit. The default was always 32, right, so they should not be suprised if they don't up the limit to 64. Now, if they modify that config.h value, they may be suprised. Is that the problem? -- Bruce Momjian | http://www.op.net/~candle maillist@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
[Charset iso-8859-1 unsupported, filtering to ASCII...] > Having recently experienced a similar problem with semaphores and > kernel size, I can say it is an issue. I feel that documentation > will clear it up either way. Either you lower the default backend > limit, and document how to raise it along with the associated kernel > variables, or leave it alone and document the appropriate steps to > tuning the kernel to accommodate it and how to lower it if you don't > want to tune the kernel. > The issue for me is that novices who are never going to hit the 32-user limit should be able to install PostgreSQL with no major changes, including not even a postmaster flag to lower the limit. If you need more than 32 connections, you will be able to modify the kernel. -- Bruce Momjian | http://www.op.net/~candle maillist@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
The default to-date has been 64. The reason you don't see much trouble with it is twofold, 1) Linux has a huge default for semaphores and shared memory, 2) The old memory model allocated semaphores in blocks of 16 up to MaxBackedId (which was hard coded to 64). I did not run into trouble on untuned Solaris until postmaster tried to start the 49th backend (semaphores 49-64 when my kernel defaulted to 60). The new model allocates semaphores and shared memory at startup, assuring you won't experience midstream troubles like I did. It does, however allocate to the max or -N setting, which most users will probably never reach. I'd vote for 32 as the default. DwD > -----Original Message----- > From: Bruce Momjian [mailto:maillist@candle.pha.pa.us] > Sent: Monday, February 22, 1999 11:57 AM > To: Daryl W. Dunbar > Cc: tgl@sss.pgh.pa.us; pgsql-hackers@postgreSQL.org > Subject: Re: [HACKERS] Re: Max backend limits cleaned up > > > [Charset iso-8859-1 unsupported, filtering to ASCII...] > > Having recently experienced a similar problem with > semaphores and > > kernel size, I can say it is an issue. I feel that > documentation > > will clear it up either way. Either you lower the > default backend > > limit, and document how to raise it along with the > associated kernel > > variables, or leave it alone and document the > appropriate steps to > > tuning the kernel to accommodate it and how to lower it > if you don't > > want to tune the kernel. > > > > The issue for me is that novices who are never going to > hit the 32-user > limit should be able to install PostgreSQL with no major changes, > including not even a postmaster flag to lower the limit. > > If you need more than 32 connections, you will be able to > modify the > kernel. > > -- > Bruce Momjian | > http://www.op.net/~candle > maillist@candle.pha.pa.us > | (610) 853-3000 > + If your life is a hard drive, | 830 Blythe Avenue > + Christ can be your backup. | Drexel Hill, > Pennsylvania 19026 >
Bruce Momjian <maillist@candle.pha.pa.us> writes:
>> Yeah, I was thinking about that myself. I left the default -N setting
>> at 64 on the theory that people who had gone to the trouble of making
>> sure they had proper kernel configurations should not get surprised by
>> v6.5 suddenly reducing the default number-of-backends limit.
> The default was always 32, right, so they should not be suprised if they
> don't up the limit to 64.
No, the default MaxBackendId was 64 (unless that's been changed
recently?). A stock installation of 6.4 will support 64 backends
assuming that you have adequate kernel settings for that.
regards, tom lane
[Charset iso-8859-1 unsupported, filtering to ASCII...] > The default to-date has been 64. The reason you don't see much > trouble with it is twofold, 1) Linux has a huge default for > semaphores and shared memory, 2) The old memory model allocated > semaphores in blocks of 16 up to MaxBackedId (which was hard coded > to 64). I did not run into trouble on untuned Solaris until > postmaster tried to start the 49th backend (semaphores 49-64 when my > kernel defaulted to 60). > > The new model allocates semaphores and shared memory at startup, > assuring you won't experience midstream troubles like I did. It > does, however allocate to the max or -N setting, which most users > will probably never reach. > > I'd vote for 32 as the default. I understand now. The stuff is preallocated. -- Bruce Momjian | http://www.op.net/~candle maillist@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
>> I am thinking, though, that we also ought to have FAQ entries under
>> headings like:
>> I get "IpcSemaphoreCreate: semget failed (No space left on device)"
>> when I try to start the postmaster
>> I get 'Sorry, too many clients already' when trying to connect
>>
>> If you like, I'll try to write up a first cut at these.
> Sure.
Attached are some proposed diffs against the copy of the FAQ that's
currently in CVS ... not sure if that is the master copy or not.
regards, tom lane
*** FAQ~ Sat Mar 6 17:10:05 1999
--- FAQ Sat Mar 6 18:00:22 1999
***************
*** 59,66 **** 2.10) All my servers crash under concurrent table access. Why? 2.11) How do I tune the database
enginefor better performance? 2.12) What debugging features are available in PostgreSQL?
! 2.13) How do I enable more than 64 concurrent backends?
! 2.14) What non-unix ports are available? Operational questions
--- 59,67 ---- 2.10) All my servers crash under concurrent table access. Why? 2.11) How do I tune the database
enginefor better performance? 2.12) What debugging features are available in PostgreSQL?
! 2.13) When I try to start the postmaster, I get IpcSemaphoreCreate errors.
! 2.14) I get 'Sorry, too many clients' when trying to connect.
! 2.15) What non-unix ports are available? Operational questions
***************
*** 384,391 **** You either do not have shared memory configured properly in kernel or you need to enlarge the
sharedmemory available in the kernel. The exact amount you need depends on your architecture and how many
! buffers you configure postmaster to run with. For most systems, with
! default buffer sizes, you need a minimum of ~760K. 2.7) I have changed a source file, but a recompile does
notsee the change?
--- 385,393 ---- You either do not have shared memory configured properly in kernel or you need to enlarge the
sharedmemory available in the kernel. The exact amount you need depends on your architecture and how many
! buffers and backend processes you configure postmaster to run with.
! For most systems, with default numbers of buffers and processes, you
! need a minimum of ~1MB. 2.7) I have changed a source file, but a recompile does not see the change?
***************
*** 420,426 **** If you are doing a lot of inserts, consider doing them in a large batch using the copy command.
Thisis much faster than single individual inserts. Second, statements not in a begin work/commit
! transaction block are considered to be their in their own transaction. Consider performing several statements
ina single transaction block. This reduces the transaction overhead. Also consider dropping and recreating
indiceswhen making large data changes.
--- 422,428 ---- If you are doing a lot of inserts, consider doing them in a large batch using the copy command.
Thisis much faster than single individual inserts. Second, statements not in a begin work/commit
! transaction block are considered to be in their own transaction. Consider performing several statements in a
singletransaction block. This reduces the transaction overhead. Also consider dropping and recreating indices
whenmaking large data changes.
***************
*** 482,494 **** pgsql/data/base/dbname directory. The client profile file will be put in the current directory.
! 2.13) How do I enable more than 64 concurrent backends?
!
! Edit include/storage/sinvaladt.h, and change the value of
! MaxBackendId. In the future, we plan to make this a configurable
! prameter.
! 2.14) What non-unix ports are available? It is possible to compile the libpq C library, psql, and other
interfacesand binaries to run on MS Windows platforms. In this case,
--- 484,530 ---- pgsql/data/base/dbname directory. The client profile file will be put in the current directory.
! 2.13) When I try to start the postmaster, I get IpcSemaphoreCreate errors.
!
! If the error message is "IpcSemaphoreCreate: semget failed (No space left
! on device)" then your kernel is not configured with enough semaphores.
! Postgres needs one semaphore per potential backend process. A temporary
! solution is to start the postmaster with a smaller limit on the number of
! backend processes (use -N with a parameter less than its default, 32).
! A more permanent solution is to increase your kernel's SEMMNS and SEMMNI
! parameters.
!
! If the error message is something else, you might not have semaphore
! support configured in your kernel at all.
!
! 2.14) I get 'Sorry, too many clients' when trying to connect.
!
! You need to increase the postmaster's limit on how many concurrent backend
! processes it can start.
!
! In Postgres 6.5, the default limit is 32 processes. You can increase it
! by restarting the postmaster with a suitable -N value. With the default
! configuration you can set -N as large as 1024; if you need more, you'll
! need to increase MAXBACKENDS in include/config.h and rebuild Postgres.
! You can set the default value of -N at configuration time, if you like,
! using configure's --with-maxbackends switch.
!
! Note that if you make -N larger than 32, you should consider increasing
! -B beyond its default of 64. For large numbers of backend processes,
! you are also likely to find that you need to increase various Unix kernel
! configuration parameters. Things to check include the maximum size of
! shared memory blocks (SHMMAX), the maximum number of semaphores (SEMMNS and
! SEMMNI), the maximum number of processes (NPROC), the maximum number of
! processes per user (MAXUPRC), and the maximum number of open files (NFILE
! and NINODE). The main reason that Postgres has a limit on the number of
! allowed backend processes is so that you can ensure that your system
! won't run out of resources.
!
! In Postgres versions prior to 6.5, the maximum number of backends was
! 64, and changing it required a rebuild after altering the MaxBackendId
! constant in include/storage/sinvaladt.h.
! 2.15) What non-unix ports are available? It is possible to compile the libpq C library, psql, and other
interfacesand binaries to run on MS Windows platforms. In this case,
> >> I am thinking, though, that we also ought to have FAQ entries under > >> headings like: > >> I get "IpcSemaphoreCreate: semget failed (No space left on device)" > >> when I try to start the postmaster > >> I get 'Sorry, too many clients already' when trying to connect > >> > >> If you like, I'll try to write up a first cut at these. > > > Sure. > > Attached are some proposed diffs against the copy of the FAQ that's > currently in CVS ... not sure if that is the master copy or not. Applied. The main file is kept as HTML for the web site, while I convert it to ASCII for the distribution. Let me know how it looks. I did a little html cleanup. You can see it on the web site. -- Bruce Momjian | http://www.op.net/~candle maillist@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026