Re: FD_SETSIZE with large #s of files/ports in use
От | Giles Lean |
---|---|
Тема | Re: FD_SETSIZE with large #s of files/ports in use |
Дата | |
Msg-id | 20100520172525.5472.qmail@sapphire.netherstone.net обсуждение исходный текст |
Ответ на | Re: FD_SETSIZE with large #s of files/ports in use (Tom Lane <tgl@sss.pgh.pa.us>) |
Список | pgsql-odbc |
Tom Lane <tgl@sss.pgh.pa.us> wrote: > "B. Nicholson" <b.nicholson@niceng.com> writes: > > Tom, what libc details will be broken by setting FD_SETSIZE to a larger > > number? I'm curious for my own knowledge base. I can see where it > > might cause 'data' sizes to grow which might break thinks. Anything else? > > I'm not too sure, honestly. I can tell you that this exact point came up > recently on a Red Hat internal mailing list, and no less an authority > than Ulrich Drepper said "you can't do that, it'll break things". He > didn't say exactly what though. It's possible that on non-glibc-based > platforms, you could get away with it. I'd guess that as FD_SETSIZE is a macro used at compile time (including compile time of libc) and that without jumping through hoops in the implementation changing it later will cause inconsistencies between the size of structures or arrays passed between the application and libc. At the risk of topic drift and providing more information than people want to know (but think of the archives! :-), here is some additional information. Summary: a) you can't rely on changing FD_SETSIZE for select(2) b) poll(2) is preferable to select(2) for performance c) interfaces that should perform better than either select(2) and poll(2) are: i. /dev/poll (Solaris, HP-UX) ii. epoll (Linux) iii. pollset (AIX) iv. kqueue (*BSDs) There is some hope of maintaining portability across this newer, non-standardised set of interfaces with libevent. PostgreSQL seems to use poll() if it's available in some places, and select() in others. (And I don't know about the Windows code.) For small numbers of file descriptors especially on non-hot code paths, it's not going to matter. In general it would be IMHO nice to use poll() consistently when it's available and not emulated via select(). Whether there is a performance gain to be had by using the non-portable solutions I don't know: it would be interesting to see some measurements, but I wouldn't necessarily expect so: the newer interfaces (certainly /dev/poll) were driven by the needs of high performance web servers with high numbers of connections which may be a too-different use case to PostgreSQL to see a notable benefit. Based on some micro benchmarks I did some years back (on now non-current OS releases which I shall not name) I would not assume that the relative performance of these interfaces (that is, select v. poll v. whatever alternative local enhancement has been created) would be consistent: you may find systems with relatively well performing select(2) implementations. Re point (a): For POSIX, FD_SETSIZE is not documented as being changeable by the application, implying that it shouldn't be altered by portable applications: http://www.opengroup.org/onlinepubs/009695399/basedefs/sys/select.h.html That's from POSIX a.k.a. IEEE Std 1003.1, 2004 Edition, a.k.a. the "Single Unix Specification Version 3". I'm no Linux guru, so I'll take Tom's and Ulrich Drepper's word for the behaviour there. Some operating systems _do_ allow applications to alter FD_SETSIZE, including at least HP-UX: http://docs.hp.com/en/B2355-60130/select.2.html Re point(b): It is "well known" that poll(2) is more efficient than select(2) as the sets of file descriptors don't have to be reset before each call as they are in select(2). (Sorry, no good reference to hand, and I'm sure someone's had an exception somewhere, at least when poll() was emulated via select()!). Re point(c): i. /dev/poll was introduced in Solaris 7, and added some time later to HP-UX: http://developers.sun.com/solaris/articles/polling_efficient.html http://docs.hp.com/en/B2355-60130/poll.7.html ii. Linux preferred to introduce epoll(7): http://www.kernel.org/doc/man-pages/online/pages/man4/epoll.4.html iii. IBM preferred pollset for AIX: http://publib.boulder.ibm.com/infocenter/aix/v6r1/topic/com.ibm.aix.basetechref/doc/basetrf1/pollset.htm iv. The *BSDs have developed kqueue; originally in FreeBSD but adopted by NetBSD and OpenBSD: http://www.freebsd.org/cgi/man.cgi?query=kqueue&sektion=2 http://netbsd.gw.com/cgi-bin/man-cgi?kqueue++NetBSD-5.0 http://www.openbsd.org/cgi-bin/man.cgi?query=kqueue&sektion=2 v. libevent: http://www.monkey.org/~provos/libevent/ Regards, Giles P.S. No, this isn't the nightmare. I just woke _up_ from the nightmare. :-) Now, back to sleep ...
В списке pgsql-odbc по дате отправления: