Re: PG 8.3 and server load

Phoenix Kiula wrote:
> On Wed, Aug 19, 2009 at 11:37 PM, Andy Colson<andy@squeakycode.net
> <mailto:andy@squeakycode.net>> wrote:
>
>  >
>  > Phoenix:  run top again, and hit the '1' key.  It'll show you stats
> for each
>  > cpu.  Is one pegged and the others idle?
>  >
>
>
> top - 10:38:53 up 29 days, 5 min,  1 user,  load average: 64.99, 65.17,
> 65.06
> Tasks: 568 total,   1 running, 537 sleeping,   6 stopped,  24 zombie
> Cpu0  : 17.7% us,  7.7% sy,  0.0% ni, 74.0% id,  0.7% wa,  0.0% hi,  0.0% si
> Cpu1  :  6.3% us,  5.6% sy,  0.0% ni, 84.4% id,  3.6% wa,  0.0% hi,  0.0% si
> Cpu2  :  5.6% us,  5.9% sy,  0.0% ni, 86.8% id,  1.7% wa,  0.0% hi,  0.0% si
> Cpu3  :  5.6% us,  4.0% sy,  0.0% ni, 74.2% id, 16.2% wa,  0.0% hi,  0.0% si
> Mem:   8310256k total,  8277416k used,    32840k free,    61944k buffers
> Swap:  2096440k total,    16128k used,  2080312k free,  7664224k cached
>
>  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
>
> 9922 nobody    15   0 49024  16m 7408 S  3.0  0.2   0:00.52 httpd
>
> 9630 nobody    15   0 49020  16m 7420 S  2.3  0.2   0:00.60 httpd
>
> 9848 nobody    16   0 48992  16m 7372 S  2.3  0.2   0:00.51 httpd
>
> 10995 nobody    15   0 49024  16m 7304 S  2.3  0.2   0:00.35 httpd
>
> 11031 nobody    15   0 48860  16m 7104 S  2.3  0.2   0:00.34 httpd
>
> 6701 nobody    15   0 49028  17m 7576 S  2.0  0.2   0:01.50 httpd
>
> 10996 nobody    15   0 48992  16m 7328 S  2.0  0.2   0:00.31 httpd
>
> 12232 nobody    15   0 48860  16m 7004 S  1.7  0.2   0:00.05 httpd
>
> 9876 nobody    15   0 48992  16m 7400 S  1.3  0.2   0:00.73 httpd
>
> 12231 nobody    15   0 48860  16m 6932 S  1.3  0.2   0:00.04 httpd
>
> 12233 nobody    16   0 48860  16m 6960 S  1.3  0.2   0:00.04 httpd
>
> 20315 postgres  19   0  325m 9732 9380 S  1.0  0.1   0:10.39 postmaster
>
> 31573 nobody    15   0 49024  17m 7664 S  1.0  0.2   0:03.14 httpd
>
> 7954 nobody    15   0 49032  16m 7400 S  1.0  0.2   0:01.14 httpd
>
> 9918 nobody    15   0 48956  16m 7344 S  1.0  0.2   0:00.44 httpd
>
> 12298 nobody    16   0 48860  16m 6780 S  1.0  0.2   0:00.03 httpd
>
> 6479 nobody    16   0 49040  16m 7412 S  0.7  0.2   0:01.20 httpd
>
> 7950 nobody    15   0 49020  16m 7388 S  0.7  0.2   0:00.83 httpd
>
> 7951 nobody    15   0 49032  16m 7384 S  0.7  0.2   0:01.03 httpd
>
> 9875 nobody    15   0 48948  16m 7096 S  0.7  0.2   0:00.51 httpd
>
> 9916 nobody    16   0 48860  16m 7124 S  0.7  0.2   0:00.59 httpd
>
> 10969 nobody    15   0 49036  16m 7380 S  0.7  0.2   0:00.29 httpd
>
> 11752 root      16   0  3620 1288  772 R  0.7  0.0   0:00.14 top
>
> 12309 nobody    16   0 48860  16m 6844 S  0.7  0.2   0:00.02 httpd
>
> 20676 mysql     15   0  182m  20m 2916 S  0.3  0.3   0:00.95 mysqld
>
> 20811 root      21   0 47920  14m 5872 S  0.3  0.2   0:00.71 httpd
>
> 7952 nobody    15   0 49024  16m 7524 S  0.3  0.2   0:00.96 httpd
>
> 11036 nobody    15   0 48992  16m 7320 S  0.3  0.2   0:00.36 httpd
>
> 12230 nobody    15   0 48860  16m 6956 S  0.3  0.2   0:00.01 httpd
>
> 12297 nobody    16   0 48860  16m 6932 S  0.3  0.2   0:00.01 httpd
>
> 12299 nobody    16   0 48992  16m 7120 S  0.3  0.2   0:00.01 httpd
>
> 12301 nobody    20   0 48860  16m 6816 S  0.3  0.2   0:00.01 httpd
>
> 12307 nobody    15   0 48860  16m 6880 S  0.3  0.2   0:00.01 httpd
>
>
>
>
>  > do a 'cat /proc/cpuinfo' and make sure your os is seeing all your cpus.
>  >
>
>
>
> I guess it's using all 4?

Yeah.

You aren't serving data from a shared drive (smb or nsf) are you?  You
have a bunch of httpd just sitting around doing very little.

Or do you have any php/perl/python/whatever turning around and doing
network stuff?

Check your nic's for errors (run ifconfig), check these stats:

RX packets:15606269 errors:0 dropped:0 overruns:0 frame:0
TX packets:13173940 errors:5 dropped:0 overruns:0 carrier:10
           collisions:0 txqueuelen:1000


the load average is a summary of a bunch of things, including whats
waiting on something else.  I'll bet your httpd's are sitting around
waiting on something, (its not cpu or disk, it must be something else),
which is causing the load average to spike up.

-Andy