Обсуждение: Too many open files
I am running PosgreSQL 7.1 on Redhat 6.2 Kernel 2.4.6. Under a pretty heavy load: 1000 Transactions per second 32 Open connections Everything restarts because of too many open files. I have increase my max number of open files to 16384 but this just delays the inevitable. I have tested the same scenario under Solaris 8 and it works fine. Is there anything I can do about this? Darin
Darin Fisher <darinf@pfm.net> writes: > I am running PosgreSQL 7.1 on Redhat 6.2 Kernel 2.4.6. > Under a pretty heavy load: > 1000 Transactions per second > 32 Open connections > Everything restarts because of too many open files. > I have increase my max number of open files to 16384 but this > just delays the inevitable. > I have tested the same scenario under Solaris 8 and it works > fine. Linux (and BSD) have a tendency to promise more than they can deliver about how many files an individual process can open. Look at pg_nofile() in src/backend/storage/file/fd.c --- it believes whatever sysconf(_SC_OPEN_MAX) tells it, and on these OSes the answer is likely to be several thousand. Which the OS can indeed support when *one* backend does it, but not when dozens of 'em do it. I have previously suggested that we should have a configurable upper limit for the number-of-openable-files that we will believe --- probably a GUC variable with a default value of, say, a couple hundred. No one's gotten around to doing it, but if you'd care to submit a patch... As a quick hack, you could just insert a hardcoded limit in pg_nofile(). regards, tom lane
From my /etc/rc.d/rc.local: # increase RCVBUF to optimize proxy<->backend echo 131072 > /proc/sys/net/core/rmem_max # increase maximum opened files echo 8192 > /proc/sys/fs/file-max # increase shared memory echo "100000000" > /proc/sys/kernel/shmmax Regards, Oleg On Wed, 1 Aug 2001, Tom Lane wrote: > Darin Fisher <darinf@pfm.net> writes: > > I am running PosgreSQL 7.1 on Redhat 6.2 Kernel 2.4.6. > > Under a pretty heavy load: > > 1000 Transactions per second > > 32 Open connections > > > Everything restarts because of too many open files. > > I have increase my max number of open files to 16384 but this > > just delays the inevitable. > > > I have tested the same scenario under Solaris 8 and it works > > fine. > > Linux (and BSD) have a tendency to promise more than they can deliver > about how many files an individual process can open. Look at > pg_nofile() in src/backend/storage/file/fd.c --- it believes whatever > sysconf(_SC_OPEN_MAX) tells it, and on these OSes the answer is likely > to be several thousand. Which the OS can indeed support when *one* > backend does it, but not when dozens of 'em do it. > > I have previously suggested that we should have a configurable upper > limit for the number-of-openable-files that we will believe --- probably > a GUC variable with a default value of, say, a couple hundred. No one's > gotten around to doing it, but if you'd care to submit a patch... > > As a quick hack, you could just insert a hardcoded limit in > pg_nofile(). > > regards, tom lane > > ---------------------------(end of broadcast)--------------------------- > TIP 3: if posting/reading through Usenet, please send an appropriate > subscribe-nomail command to majordomo@postgresql.org so that your > message can get through to the mailing list cleanly > Regards, Oleg _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83
Thanks, so far that looks like it is helping. Only time will tell :) I take it, that the pg_nofile is the max number of file to open per postgres session? Darin Tom Lane wrote: > Darin Fisher <darinf@pfm.net> writes: > > I am running PosgreSQL 7.1 on Redhat 6.2 Kernel 2.4.6. > > Under a pretty heavy load: > > 1000 Transactions per second > > 32 Open connections > > > Everything restarts because of too many open files. > > I have increase my max number of open files to 16384 but this > > just delays the inevitable. > > > I have tested the same scenario under Solaris 8 and it works > > fine. > > Linux (and BSD) have a tendency to promise more than they can deliver > about how many files an individual process can open. Look at > pg_nofile() in src/backend/storage/file/fd.c --- it believes whatever > sysconf(_SC_OPEN_MAX) tells it, and on these OSes the answer is likely > to be several thousand. Which the OS can indeed support when *one* > backend does it, but not when dozens of 'em do it. > > I have previously suggested that we should have a configurable upper > limit for the number-of-openable-files that we will believe --- probably > a GUC variable with a default value of, say, a couple hundred. No one's > gotten around to doing it, but if you'd care to submit a patch... > > As a quick hack, you could just insert a hardcoded limit in > pg_nofile(). > > regards, tom lane > > ---------------------------(end of broadcast)--------------------------- > TIP 3: if posting/reading through Usenet, please send an appropriate > subscribe-nomail command to majordomo@postgresql.org so that your > message can get through to the mailing list cleanly
> I take it, that the pg_nofile is the max number of file to open per postgres > session? Right, it's per backend. regards, tom lane