Обсуждение: How to monitor resources on Linux.

Hello All

    I have some questions on memory resources and linux. We are
currently running Dell Poweredge 2950 with dual core opeterons and 8GB
RAM. Postgres version is 7.4.17 on RHEL4. Could someone explain to me
how to best monitor the memory resources on this platform. Top shows a
high memory usage nearly all is being used. ipcs -m shows the following
output. If I am looking at this correctly each of the postgres entries
represents a postmaster with the number of connections. If I calculate
the first entry it comes to around 3.4GB of RAM being used is this
correct. We have started running into memory issues and I think we have
exhausted all the memory on the system. I think the best approach would
be to add more memory unless someone can suggest other options.  We have
a 2 node cluster running about 10 separate postmasters divided evenly on
each node. Each postmaster is a separate division is our company if we
have a problems with one database not everyone is down.

0x0052ea91 163845     postgres  600        133947392  26
0x00530db9 196614     postgres  600        34529280   24
0x00530201 229383     postgres  600        34529280   21
0x005305e9 262152     postgres  600        4915200    3
0x005311a1 294921     postgres  600        34529280   28
0x0052fe19 327690     postgres  600        4915200    4

Thanks

John Allgood - Systems Admin
Turbo Logistics

Hello Again             This is interesting. We are running a 32bit kernel. I will send another output from free in the
morningwhen the load on the server is greater. We have been using this equipment since April and because of another
issuewhich required reboots everynight. This is really the first time that we have had any uptime on this machine. It
wasup for 18days. Before we started having any issues. The output from free -l -m. I believe the High and Low are like
watermarksfor lack of another word. 

Thanks

John R Allgood <jallgood@the-allgoods.net> writes:
> Here is the output from free on one of the nodes.

Hmmm ... I'm not exactly a kernel jock, but I find the presence of these
lines in the output to be mighty suspicious:

> Low:           821        510        310
> High:         7294       5459       1835

In a true 64-bit system you should not have any distinction between low
and high memory (and indeed "free" doesn't print any such thing on my
x86_64 box).  Maybe you are running a 32-bit kernel?  Anyway it seems
likely that the out-of-memory situation was due to oversubscribed lowmem
rather than being out of memory globally, especially since you are
running at zero swap usage.  This is not the trace of a system that's
under any memory pressure overall:

>              total       used       free
> Mem:          8116       5969       2146
> -/+ buffers/cache:       1506       6609
> Swap:         2000          0       1999

Something else to check is whether having swap only a quarter the size
of physical RAM is a good idea or not.  I'm not sure what the latest
configuration theories are for Linux, but the old-school-Unix theory was
always that you should have more swap space than RAM.  When memory
overcommit is disabled, having plenty of swap space available may be
necessary even if you're seemingly not using it --- the kernel needs to
be sure that there would be someplace to put everything if it had to
materialize all the virtual copy-on-write pages that the current process
set is sharing.
        regards, tom lane

jallgood@the-allgoods.net writes:
>      This is interesting. We are running a 32bit kernel.

On an Opteron?  Why in the world are you doing that?

> The output from free -l -m. I believe the High and Low are like
> watermarks for lack of another word.

Uh, no, you are dead wrong.  In a 32-bit machine low memory is the first
physical GB or so, and high memory is the rest, and there are certain
things that have to be in low memory because the hardware won't cope
otherwise.  Thus, you can run out of lowmem even when there's scads of
free memory in highmem.

If you've got more than about a GB of physical RAM you need to be
running a 64-bit kernel; otherwise you're wasting your hardware.

            regards, tom lane

Hello   We have this one app "pcmiler" that ties directly into our backend of our application and will not run in an
64bitenviroment. We had to get ALK to fix some bugs in there code and that is what all the reboots were about mentioned
earlier.Put it this way pcmiler was going places in memory it didn't need to go. They did correct the problem. 

jallgood@the-allgoods.net writes:
>      This is interesting. We are running a 32bit kernel.

On an Opteron?  Why in the world are you doing that?

> The output from free -l -m. I believe the High and Low are like
> watermarks for lack of another word.

Uh, no, you are dead wrong.  In a 32-bit machine low memory is the first
physical GB or so, and high memory is the rest, and there are certain
things that have to be in low memory because the hardware won't cope
otherwise.  Thus, you can run out of lowmem even when there's scads of
free memory in highmem.

If you've got more than about a GB of physical RAM you need to be
running a 64-bit kernel; otherwise you're wasting your hardware.
        regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

jallgood@the-allgoods.net writes:
>     We have this one app "pcmiler" that ties directly into our backend
>     of our application and will not run in an 64bit enviroment.

Hmmm ... we have heard of pcmiler before on these lists, and there has
never been anything good said about it.  Do a bit of archive searching,
and then consider if you can't find something else.

I'm not totally clear on what pcmiler really does, but perhaps there's
some overlap with PostGIS?  Not that PostGIS has no bugs, but at least
it's open-source code and you can hope to get useful commentary from
people who can look at the source.  Right at the moment I think you are
stuck trying to get support from pcmiler's authors, because no one else
in the world knows everything that's happening in your system or can fix
it.

            regards, tom lane

Hello All

    Yeap that was us on the pcmiler past issues. This app provides mileage lookup for our application. Our customers require us to use that package for mileage lookup. Unfortunately there linux version is not developed as heavily as the windows version when we started having the vendor look at the code we discovered that it was compiled under Redhat 8.0. At this point I believe that maybe we don't have an "not enough memory issue" but maybe a memory bleed. This will definitely require us to dig a little deeper.

Tom Lane wrote:

jallgood@the-allgoods.net writes: 

    We have this one app "pcmiler" that ties directly into our backend   of our application and will not run in an 64bit enviroment.   

Hmmm ... we have heard of pcmiler before on these lists, and there has
never been anything good said about it.  Do a bit of archive searching,
and then consider if you can't find something else.

I'm not totally clear on what pcmiler really does, but perhaps there's
some overlap with PostGIS?  Not that PostGIS has no bugs, but at least
it's open-source code and you can hope to get useful commentary from
people who can look at the source.  Right at the moment I think you are
stuck trying to get support from pcmiler's authors, because no one else
in the world knows everything that's happening in your system or can fix
it.
		regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate      subscribe-nomail command to majordomo@postgresql.org so that your      message can get through to the mailing list cleanly
 

On Wed, 29 Aug 2007, John R Allgood wrote:
> [pcmiler] was compiled under Redhat 8.0.

I thought that you might find virtualization useful. (Run redhat 8.0 as a
guest OS) Sorry for the noise if you already have considered it, but it is
a typical case vmware used to tout as a success story. (Running legacy
software inside a legacy guest OS) My experience was with vmware and I
think it will do the job but then I haven't used others yet.

http://www-128.ibm.com/developerworks/library/l-linuxvirt/index.html#resources

My 2pence

Ben K.
Developer
http://benix.tamu.edu

On Wed, Aug 29, 2007 at 08:05:00AM -0400, John R Allgood wrote:
>    Yeap that was us on the pcmiler past issues. This app provides
> mileage lookup for our application. Our customers require us to use that
> package for mileage lookup.

I don't know anything about pcmiler, but does it have to run on the
same box as the back end?  Maybe you could put it somewhere else, and
then you could make your databases work well.  Finding a nice 32-bit
Intel box for it oughta be pretty trivial (=="cheap") these days.

A

--
Andrew Sullivan  | ajs@crankycanuck.ca
This work was visionary and imaginative, and goes to show that visionary
and imaginative work need not end up well.
        --Dennis Ritchie

I wish that was the case the linux version has to be installed locally. If you are running Winders you can connect remotely but we don't want to do that.

Andrew Sullivan wrote:

On Wed, Aug 29, 2007 at 08:05:00AM -0400, John R Allgood wrote: 

   Yeap that was us on the pcmiler past issues. This app provides 
mileage lookup for our application. Our customers require us to use that 
package for mileage lookup.    

I don't know anything about pcmiler, but does it have to run on the
same box as the back end?  Maybe you could put it somewhere else, and
then you could make your databases work well.  Finding a nice 32-bit
Intel box for it oughta be pretty trivial (=="cheap") these days.

A
 

> I wish that was the case the linux version has to be installed locally. If

> you are running Winders you can connect remotely but we don't want to do
> that.

Maybe you could use pgpool or something similar to allow it to think it's
connecting locally and still move it off the actual postgresql server?

"Jeff Frost" <jeff@frostconsultingllc.com> writes:
> Maybe you could use pgpool or something similar to allow it to think it's
> connecting locally and still move it off the actual postgresql server?

Actually, I don't see anything very wrong with running a 32-bit database
executable, so long as you don't have ambitions to have more than a GB
or so of shared_buffers.  The thing that's not good is a 32-bit kernel.

            regards, tom lane

On Wed, 29 Aug 2007, John R Allgood wrote:

> I wish that was the case the linux version has to be installed locally.

I'm not an expert but I guess "local" may be a bit different in
virtualization.

If "local" requirement is from license or security issues, there are ways
to completely hide the guest from outside networks (provide it only with a
host-to-guest (virtual) network)  and it may qualify for local, looking
from the outside.

If it is from technical requirement and means non-tcp/ip, I guess there
may be programmatic solutions possible between host and guest.
(http://pubs.vmware.com/vmci-sdk/VMCI_intro.html)

Just thought it might be worth exploring.

HTH.


Regards,

Ben K.
Developer
http://benix.tamu.edu

can it possiable to  uses Postgresql in windows. From where i will verions
for that





>From: Ben Kim <bkim@tamu.edu>
>To: John R Allgood <jallgood@the-allgoods.net>
>CC: pgsql-admin@postgresql.org
>Subject: Re: [ADMIN] How to monitor resources on Linux.
>Date: Fri, 31 Aug 2007 08:17:12 -0500 (CDT)
>
>On Wed, 29 Aug 2007, John R Allgood wrote:
>
>>I wish that was the case the linux version has to be installed locally.
>
>I'm not an expert but I guess "local" may be a bit different in
>virtualization.
>
>If "local" requirement is from license or security issues, there are ways
>to completely hide the guest from outside networks (provide it only with a
>host-to-guest (virtual) network)  and it may qualify for local, looking
>from the outside.
>
>If it is from technical requirement and means non-tcp/ip, I guess there may
>be programmatic solutions possible between host and guest.
>(http://pubs.vmware.com/vmci-sdk/VMCI_intro.html)
>
>Just thought it might be worth exploring.
>
>HTH.
>
>
>Regards,
>
>Ben K.
>Developer
>http://benix.tamu.edu
>
>---------------------------(end of broadcast)---------------------------
>TIP 7: You can help support the PostgreSQL project by donating at
>
>                http://www.postgresql.org/about/donate

_________________________________________________________________
Real Estate classifieds on MSN - for free.www.yello.in
http://www.yello.in/home.php?utm_source=hotmailtag&utm_medium=email&utm_content=in&utm_campaign=jun

>>> On Fri, Aug 31, 2007 at  8:51 AM, in message
<BAY116-F382360C477D966D3137431BBCE0@phx.gbl>, "Jayaram Bhat"
<bhatindia@hotmail.com> wrote:
> can it possiable to  uses Postgresql in windows. From where i will verions
> for that

That was rather off-topic for the thread -- you should have started a new
 one.

Anyway, if I understand your post, you're looking for this:

http://www.postgresql.org/ftp/binary/v8.2.4/win32/

-Kevin