Re: Memory issue on FreeBSD

Since the top reporting goes back to normal when postgresql is stopped ,
and since postgresql is special due to the use of IPC, i would be inclined
to think that the culprit here is the shared memory.

I don't know where maintenance_work_mem really lives (process normal address space or IPC shared mem)
and if that makes any difference. If it is possible you might play with those two values and see if anything changes.

Currently i have :

maintenance_work_mem = 960MB # pgtune wizard 2012-11-01
shared_buffers = 3840MB # pgtune wizard 2012-11-01

top:
last pid: 74896;  load averages:  0.02,  0.08,  0.08                                                          up
4+06:20:31 18:14:19 
187 processes: 1 running, 172 sleeping, 14 zombie
CPU:     % user,     % nice,     % system,     % interrupt,     % idle
Mem: 4064M Active, 8111M Inact, 2014M Wired, 322M Cache, 1645M Buf, 1106M Free
Swap: 8000M Total, 608K Used, 7999M Free

hw.physmem: 17144205312
hw.usermem: 15028662272
hw.realmem: 17985175552

top (excluding Buf) amounts to 15617 Megs while physmem shows as 16349 Megs

but as i said i run 8.3 on AMD64 and pgsql 9.2.1

On Δευ 05 Νοε 2012 16:11:39 Frank Broniewski wrote:
> Hi,
>
> I just add the different memory values together (minus the buffers).
> Usually this sums up (+/-) to the installed memory size, at least on my
> other machines. I found a thread similar to my problem here [1], but no
> solution. I don't mind top showing false values, but if there's a larger
> problem behind this, then I really want to solve it.
>
> Top is really just an indicator for this issue, it's also visible in my
> munin stats [2]
>
> Below is a output _without_ postgresql running:
> Mem: 59M Active, 17G Inact, 3953M Wired, 1325M Cache, 3283M Buf, 8663M Free
> Swap: 4096M Total, 828K Used, 4095M Free
>
>
> and this is after a few hours of running:
>
> Mem: 91M Active, 17G Inact, 3983M Wired, 1526M Cache, 3283M Buf, 155M Free
> Swap: 4096M Total, 828K Used, 4095M Free
>
> some memory related sysctl values:
> hw.realmem: 34879832064
> hw.physmem: 34322804736
> hw.usermem: 30161108992
>
> # sysctl vm.vmtotal
> vm.vmtotal:
> System wide totals computed every five seconds: (values in kilobytes)
> ===============================================
> Processes:        (RUNQ: 1 Disk Wait: 0 Page Wait: 0 Sleep: 70)
> Virtual Memory:        (Total: 1084659688K Active: 10400940K)
> Real Memory:        (Total: 1616176K Active: 1349052K)
> Shared Virtual Memory:    (Total: 60840K Active: 14132K)
> Shared Real Memory:    (Total: 11644K Active: 8388K)
> Free Memory Pages:    7263972K
>
>
> [1]
> http://lists.freebsd.org/pipermail/freebsd-stable/2011-January/061247.html
> [2]
> http://www.gis-hosting.lu/monitor/munin/metrico/bilbo.metrico/memory.html
>
>
> Am 2012-11-05 15:21, schrieb Achilleas Mantzios:
> > How do you measure that smth is missing from top? What values do you add?
> > I am currently running 8.3 but we shouldn't be so far apart top-wise.
> > What is the reading under SIZE and RES in top for all postgresql processes?
> > Take note that shared mem should be recorded for each and every postmaster running.
> >
> > On Δευ 05 Νοε 2012 14:36:44 Frank Broniewski wrote:
> >> Hi,
> >>
> >> thank you for your feedback. I had a look at those commands and their
> >> output, especially in conjunction with the SEGSZ value from icps  -am
> >>
> >> Here's an example output:
> >> # ipcs -am
> >> Shared Memory:
> >> T           ID          KEY MODE        OWNER    GROUP    CREATOR
> >> CGROUP         NATTCH        SEGSZ         CPID         LPID ATIME
> >> DTIME    CTIME
> >> m       262144      5432001 --rw------- pgsql    pgsql    pgsql    pgsql
> >>                12   8813993984        45512        45512 13:49:28
> >> 14:31:29 13:49:28
> >>
> >> but frankly this tells me nothing. I can tell that the value SEGSZ is
> >> right from the start 8813993984 and it doesn't change anymore. The only
> >> value that changes is the NATTCH value, I observed a range from 8 to 36
> >> there. I agree that the SEGSZ value matches the 8GB shared buffer, but
> >> how can I make the connection of my 5GB missing in top? I wonder if this
> >> might be the maintenance_work_mem, which is set to 4GB?
> >>
> >> Many thanks,
> >>
> >> Frank
> >>
> >> Am 2012-11-05 12:14, schrieb Achilleas Mantzios:
> >>>
> >>> ipcs in FreeBSD is a little ... tricky.
> >>>
> >>> ipcs -M
> >>> ipcs -m
> >>> ipcs -am
> >>>
> >>> could be your friends
> >>>
> >>> On Δευ 05 Νοε 2012 11:22:46 Frank Broniewski wrote:
> >>>> Hi,
> >>>>
> >>>> I am running a PostgreSQL server on FreeBSD. The system has 32GB memory.
> >>>> Usually I use top to examine the memory usage of the system. After a
> >>>> while, a part, approximately 5GB, vanish from top, so that the memory
> >>>> rounds up to 27GB.  After restarting PostgreSQL, I have all 32GB again
> >>>> available, but then it's already slightly decreasing. It's a standalone
> >>>> database server. It has an OpenStreetMap world database running with
> >>>> 353GB data (with indices).
> >>>>
> >>>> Some system information:
> >>>> # uname -r
> >>>> 9.0-RELEASE-p3
> >>>> # pg_ctl --version
> >>>> pg_ctl (PostgreSQL) 9.1.6
> >>>>
> >>>> # cat /boot/loader.conf
> >>>> ...
> >>>> kern.ipc.semmni=256
> >>>> kern.ipc.semmns=512
> >>>> kern.ipc.semmnu=256
> >>>> kern.ipc.semumr=200
> >>>> vm.pmap.shpgperproc=400
> >>>> vm.pmap.pv_entry_max=50331648
> >>>> ...
> >>>>
> >>>> # cat /pgdata/data/postgresql.conf
> >>>> ...
> >>>> default_statistics_target = 50 # pgtune wizard 2012-04-04
> >>>> maintenance_work_mem = 4GB # pgtune wizard 2012-04-04
> >>>> constraint_exclusion = on # pgtune wizard 2012-04-04
> >>>> checkpoint_completion_target = 0.9 # pgtune wizard 2012-04-04
> >>>> effective_cache_size = 24GB # pgtune wizard 2012-04-04
> >>>> work_mem = 768MB # pgtune wizard 2012-04-04
> >>>> wal_buffers = 16MB # pgtune wizard 2012-04-04
> >>>> checkpoint_segments = 60 # 20
> >>>> shared_buffers = 8GB # pgtune wizard 2012-04-04
> >>>> max_connections = 100
> >>>> synchronous_commit = off
> >>>>
> >>>>
> >>>> So any help finding out why my system "looses" some RAM is greatly
> >>>> appreciated :-) If more information is needed I will gladly provide it.
> >>>>
> >>>> Frank
> >>>>
> >>>>
> >>>>
> >>>>
> >>> -
> >>> Achilleas Mantzios
> >>> IT DEPT
> >>>
> >>>
> >>
> >>
> >>
> > -
> > Achilleas Mantzios
> > IT DEPT
> >
> >
>
>
>
-
Achilleas Mantzios
IT DEPT