Philip Warner wrote:
> At 13:49 16/03/01 -0500, Jan Wieck wrote:
> >
> > Similar problem as with shared memory - size. If a long
> > running backend of a multithousand table database needs to
> > send access stats per table - and had accessed them all up to
> > now - it'll be alot of wasted bandwidth.
>
> Not if you only send totals for individual counters when they change; some
> stats may never be resynced, but for the most part it will work. Also, does
> Unix allow interrupts to occur as a result of data arrivibg in a pipe? If
> so, how about:
>
> - All backends to do *blocking* IO to collector.
The general problem remains. We only have one central collector with a limited receive capacity. The more
load is on the machine, the smaller it's capacity gets. The more complex the DB schemas get and the more
load is on the system, the more interesting accurate statistics get. Both factors are contraproductive. More
complexschema means more tables and thus bigger messages. More load means more messages. Having good
statisticson a toy system while they get worse for a web backend server that's really under pressure is
braindeadfrom the start.
We don't want the backends to block, so that they can do THEIR work. That's to process queries, nothing
else.
Pipes seem to be inappropriate because their buffer is limited to 4K on Linux and most BSD flavours.
Message queues are too because they are limited to 2K on most BSD's. So only sockets remain.
If we have multiple processes that try to receive from the UDP socket, condense the received packets into
summary messages and send them to the central collector, this might solve the problem.
Jan
--
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================== JanWieck@Yahoo.com #
_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com