Re: Performance and Clustering

> I dont think its that easy.  50,000 sockets open, sure, but whats the
> performance?  The programming model has everything to do with that,
> and windows select() wont support that many sockets with any sort of
> performance.  For windows you have to convert to using non-blocking
> sockets w/messages.  (and I've never see the PG code, but I'll bet
> it's not using non-blocking sockets & windows msg q, so 50k sockets
> using select() on windows will not be usable).
>
> That being said, I'm not a windows socket component developer, so its
> mostly guessing.  But saying "it can" and saying "its usable" are two
> different things, and that depends on the code, not the registry
> settings.
Actually that is incorrect. You can use Synchronous non-blocking
sockets. Asynchronous is a nightmare due to the overhead of pushing and
handling messages... the busier the kernel, the slower your application.
Syn-Non-Blocking will perform a small degradation in performance every
5,000 sockets. (Meaning 10,000 streams is minimally slower than 5,000 -
but enough to denote degradation).

Systems Running my product and Designs:

     AOL's Proxy Server System
     Some of the UK's largest ISP's
     AT&T Fiber Monitoring Framework
     HBO Video Streaming to Satellite
     Hart, a Front-End for TransUnion, Equifax and Experian
     OFAC Query (B-Tree Query Service, processing over 100,000 requests
a second) (*)

* WAN Latency plays a running variable on their stats, but they average
100,000+ a second during peak-hours. [1 master, 2 fail-over
load-balanced servers].

Most people run into the "2048+/-" thread limitation until they learn
how to properly manage stack allocation per thread. I have been
designing commercial enterprise socket solutions for over 15 years and
sell an SDK that no product has yet to touch and I compete with ALL the
big boys (and they all know who I am). :-) ... the limitations in
performance are factors of poor (modern sloppiness) variable allocation,
memory management, buffering techniques, etc. I got out of actively
promoting DXSock (my socket suite) when I found I could capitalize more
on my time and my product... so since 2000 - I sale my knowledge.

Factors which also come into play are the built-in overhead of the
Operating System when it is a "Network Client/Server" it has active
connections. These connections also incur the poor default settings
Microsoft picked (FIN_WAIT/2 issue which is another registry tweak).
Once you learn what servers a "Dedicated Windows Server" will not need,
rip out all of the excess "Network Client" junk (and this is well
documented all over the net) - you can produce very robust Windows
servers. (Of course there are much better solutions for production
servers than Windows - but, people still drink the Microsoft "blue"
coolaide.

* People who document the registry tweaks needed:
http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.dc38421_1500/html/ntconfig/X26667.htm

;-)

O.