Re: scaling up postgres

> One beast will be apache, and the other will be postgres.

> I'm using httperf/autobench for measurments and the best result I can get
> is that my system can handle a trafiic of almost 1600 New con/sec.

> NB : apache when stressed for a static page, i can handle more 16k new
> con/sec

    That's not the point.
    Here are a few points of advice.

    USE LIGHTTPD DAMMIT !

    If you want performance, that is.
    On my server (Crap Celeron) it handles about 100 hits/s on static files
and 10 hits/s on PHP pages ; CPU utilization is 1-2%, not counting PHP.
    lighttpd handles 14K static pages/s on my laptop. That's about as much as
your bi-xeon does with apache...

    You want a web server that uses as little CPU as possible so that more
CPU is left for generating dynamic content.

    Also, you want to have a number of concurrent DB connections which is
neither too high, nor too low.
    Apache + mod_php needs to spawn a lot of processes, thus you need a lot
of database connections.
    This tends not to be optimal.

    Too few concurrent DB connections -> network latency between DB and
webserver will be a bottleneck.
    Too many connections -> excess context switching, suboptimal use of CPU
cache, memory use bloat.

    So, I advise you to use lighttpd fronting PHP as fastcgi (if you use PHP)
; if you use Java or whatever which has a HTTP interface, use lighttpd as
a proxy for your dynamic page generation.
    Spawn a reasonable number of PHP processes. The number depends on your
application, but from 10 to 30 is a good starting point.

    USE PERSISTENT DATABASE CONNECTIONS !

    Postgres will breathe a little better ; now, check if it is still slow.
If it is, you need to find the bottleneck...
    I can help you a bit by private email if you want.