Re: How to monitor resources on Linux.

Here is the output from free on one of the nodes. I have seen free mem go as low as 15 and then go back up. Like I was saying earlier my concern was why the kernel started killing my postmasters. Here is the kernel message "kernel: oom-killer: gfp_mask=0xd0". This started happening when were running our midday backup. After backup runs vacuum will start up right after. We rebooted the servers last night and today backup and vacuum ran fine. Below the free out put I have added the logging out of /var/log/messages. Thanks this is a great list.

i                             total       used       free     shared    buffers     cached
Mem:          8116       5969       2146          0        144       4318
Low:           821        510        310
High:         7294       5459       1835
-/+ buffers/cache:       1506       6609
Swap:         2000          0       1999


ug 27 12:24:42 gan-lxc-01 kernel: Swap cache: add 2104, delete 2017, find 829/1136, race 0+0
Aug 27 12:24:42 gan-lxc-01 kernel: 1229 bounce buffer pages
Aug 27 12:24:42 gan-lxc-01 kernel: Free swap:       2047424kB
Aug 27 12:24:42 gan-lxc-01 kernel: 2260992 pages of RAM
Aug 27 12:24:42 gan-lxc-01 kernel: 1867512 pages of HIGHMEM
Aug 27 12:24:42 gan-lxc-01 kernel: 183273 reserved pages
Aug 27 12:24:42 gan-lxc-01 kernel: 942026 pages shared
Aug 27 12:24:42 gan-lxc-01 kernel: 87 pages swap cached
Aug 27 12:24:42 gan-lxc-01 kernel: Out of Memory: Killed process 19383 (postmaster).



Scott Marlowe wrote:

On 8/28/07, Andrew Sullivan <ajs@crankycanuck.ca> wrote: 

On Tue, Aug 28, 2007 at 03:40:03PM -0400, John R Allgood wrote:   

lot of activity as compared to the other databases. We run VACUUM at
midday VACUUM FULL at night, VACUUM ANALYZE on weekends.     

If you are running VACUUM often enough, then you should _never_ need
VACUUM FULL.  And weekly VACUUM ANALYSE is probably too infrequent.   

I would go so far as to say that vacuum fulls should never need to be
scheduled.  they should only be run when the DBA has looked at the DB
and determined that "something bad has happened" and needs to run it.
And even then, reindexdb is usually a better choice.

Also, by 7.4 autovacuum existed, even if it isn't perfect yet.  It's
still better than weekly analyze.

As for the top output, I'm pretty sure it's in bytes.

133947392 is about 125Meg as the OP mentioned later is what he has
shared mem set to.

You said: "we see memory usage peak and then it will go down"

What do you mean by this?  What does free say before during and after.

Here's free on my db server right now:
            total       used       free     shared    buffers     cached
Mem:       2072460    2043440      29020          0      42980    1891160
-/+ buffers/cache:     109300    1963160
Swap:      2097144        536    2096608

Note that I'm showing 29Meg free.  But I've got 42Meg buffers and 1.8Gig cached.

My memory's not used up.

So, we're all just trying to be sure that you really are running out of memory.

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?
              http://www.postgresql.org/docs/faq