Обсуждение: Low level socket locking...

Hi

I am writing a C program to work as an interface between postgres and a
web page. This page will get a LOT of activity.

The system is to be set up like this:

                 Web page, CGI generated
                        |
         Connection to handler process via unix socket
                        |
     Handler process, that has a postgres connection open.

This is set up in this way because I need to do many similar jobs with the
database, but obviously, every time the web page is accessed, a new copy of
the cgi script is created, and if I were to create a new connection to the
database every time this would a) mean spawning LOTS of copies of the
postgres backend, very slow, and b) I wouldnt get the advantages of any
caching that is done.
Thus I send a simple message from the CGI to the handler process, which
then interracts with the database, and returns the result.

Now, my problem is this.

The page being accessed is likely, in time, to get enough hits that as the
jobs are handled one at a time by the handler process (by the nature of the
postgres C librray functions), it is very feasable that a backlog could form.

I was planning on simply fork()ing the handler process. The database would be
open on each child, and I could happily process them all easilly. However
it occurred to me that I do not know how the socket code between the
postgres libraries and the postgres backend is handled? Is there a socket
locking process in place, to prevent two children passing data down the same
socket at the same time, thus causing garbled data to be received at the other
end? If I fork() and send data from one of say, three child processes, am
I guarenteed to get the data back at the same child?
(this all assumes that I open the database connection and then fork() after)


Thanx in advance for any info

 =>From: M Simms <grim@argh.demon.co.uk>
 =>...
 =>I am writing a C program to work as an interface between postgres and a
 =>web page. This page will get a LOT of activity.
 =>...
 =>I was planning on simply fork()ing the handler process. The database would be
 =>open on each child, and I could happily process them all easilly. However
 =>it occurred to me that I do not know how the socket code between the
 =>postgres libraries and the postgres backend is handled? Is there a socket
 =>locking process in place, to prevent two children passing data down the same
 =>socket at the same time, thus causing garbled data to be received at the other
 =>end? If I fork() and send data from one of say, three child processes, am
 =>I guarenteed to get the data back at the same child?
 =>(this all assumes that I open the database connection and then fork() after)

I've written a similar program for Oracle and Sybase, and (while I
can't speak directly to the question of whether you can do what you
want in PostgreSQL), I'd strongly suggest opening the DB connection
*after* you fork().  You won't have to worry about multiple children
confusing the back end, and it'll work for other databases too, if you
need that capability later.  The sequence you want to do is:

* Prepare to become a daemon (lose controlling TTY, chdir("/"), close
  all open descriptors, etc)
* Open the socket you're going to receive requests on.
* Fork() as many children as you need
> In each child, connect to the database, then do
  accept/read/write/close on the socket to respond to the client's
  requests
> In the parent, watch for children dying, and spawn new ones to
  replace them.

You can make this even nicer by having the parent & children collude
to do load balancing -- if all the children have clients, the parent
should start some more.  If many children are idle for a while, they
should die off.  You can even have the parent serve requests on a
"control channel" to change the number of children or to retrieve
statistics about the daemon's activity.

d.

M Simms wrote:
>
> This is set up in this way because I need to do many similar jobs with the
> database, but obviously, every time the web page is accessed, a new copy of
> the cgi script is created, and if I were to create a new connection to the
> database every time this would a) mean spawning LOTS of copies of the
> postgres backend, very slow, and b) I wouldnt get the advantages of any
> caching that is done.
> Thus I send a simple message from the CGI to the handler process, which
> then interracts with the database, and returns the result.

AFAIK, you have three possibilities:

- Write a daemon who waits for connections from the CGI and does all the
database-related stuff. This is slightly hard to implement but the most flexible
solution, probably not the best thought...

- Use FastCGI (http://fastcgi.idle.com), it's very similar to the first
solution, since the fastcgi module will spawn one or more persistent processes
and will dispatch the requests to them thru sockets (eventually, you may move
the processes on a different machine).
You would open the database connection only once when the process is spawned.
Advantages: Very fast as the processes may be written in C.

- Use PHP3 with pg_pConnect (persistent Connect), the PHP module will mantain a
cache of open connections and efficently reuse each of them.

Hope this helps...

Bye!

--
 Daniele

-------------------------------------------------------------------------------
"Oh, I've seen copies [of Linux Journal] around the terminal room at The
Labs."
(By Dennis Ritchie)
-------------------------------------------------------------------------------
 Se telecom italia aggiungesse uno scatto al giorno ad ogni abbonato, dal
 primo Gennaio avrebbe rubato 682.040.000.000 Lire.
-------------------------------------------------------------------------------
 Daniele Orlandi - Utility Line Italia - http://www.orlandi.com
 Via Mezzera 29/A - 20030 - Seveso (MI) - Italy
-------------------------------------------------------------------------------