Обсуждение: OOM on large SELECT

Angelo Nicolosi wrote:

> The dmesg command says:
>
> Out of memory: Killed process 22592 (postgres).

If you are getting OOM-kills from the kernel, that usually means you
have system-wide memory exhaustion.

> I wrote some C-Language functions, added on a database [...]

Since the size of a JVM is roughly constant, I would guess that the
problem actually lies in your C code.

Alternatively your system just doesn't have enough virtual memory
configured to do whatever it is you're trying to do. Add more swap.

Either way, it doesn't look like a JDBC driver problem based on the
information you have given.

-O

Angelo Nicolosi wrote:
> It's possible that the problem is in my C code but every time that I'm
> allocating memory, using always the palloc() function, I'm always
> calling the pfree().
> There is some way to analyze the code meanwhile is working inside the
> Postgre server (something like valgrind)?
> However the command free -m on my machine outputs:
>
>              total       used       free     shared    buffers     cached
> Mem:          2010        664       1345          0        157        383
> -/+ buffers/cache:        123       1886
> Swap:        16386         41      16345
>
> I think that the swap is enough.
> Could you give me some tips about how can I see where is the problem?
> Thank you for your help!

do you know what query you were making when you ran out of memory?   it
-appears- it was a postgres server process that was OOM'd.

what OS and version are you on (OOM seems to imply its likely linux,
since no other OS I'm familiar with would randomly kill processes like
that), what version postgres, etc ?

also, what are the various memory settings in your postgresql.conf
(shared_buffers, work_mem, etc)

Angelo Nicolosi wrote:

> There is some way to analyze the code meanwhile is working inside the
> Postgre server (something like valgrind)?

You are probably better off asking on pgsql-hackers.

-O

Angelo Nicolosi wrote:
> Sorry for the delay of this answer but i was trying to figure out.
> However I saw that the memory that the postgres is using is getting
> larger step by step.
> So it doesn't free it.
> After the third query it is already full and one of the thread of the
> postgres is killed from the OOM.
> When the process is killed the program usually is going to call again
> a stored function.
> By the way the info that you required me are:

I wonder if the work_mem settings in postgresql.conf are too large for
this computer?   if you're not the postgres administrator, you'll need
to work with them to identify this problem.

note, btw, 8.4.0 has been superceded by 8.4.1, with a lot of bug
fixes.   painless inplace upgrade.

On Thu, 2009-09-17 at 19:03 +0200, Angelo Nicolosi wrote:
> Sorry for the delay of this answer but i was trying to figure out.
> However I saw that the memory that the postgres is using is getting
> larger step by step.
> So it doesn't free it.

If the memory is allocated using palloc() and not freed even after the
query finishes, then you must be using a wrong memory context.

> After the third query it is already full and one of the thread of the
> postgres is killed from the OOM.
> When the process is killed the program usually is going to call again
> a stored function.
> By the way the info that you required me are:
>
>
> postgres (PostgreSQL) 8.4.0
> Linux kernel 2.6.18 64bits
>
>
> For the memory settings I have to contact the system admin because i
> don't have the rights, on that machines, to read the configurations
> file.

do

show work_mem;

from psql;

to see all conf params, do

show all;



> Thank you again to all for your help.
> Cheers,
> Angelo.
>
> > Date: Wed, 16 Sep 2009 09:30:59 -0700
> > From: pierce@hogranch.com
> > To: amenuor@hotmail.com
> > CC: pgsql-jdbc@postgresql.org
> > Subject: Re: [JDBC] OOM on large SELECT
> >
> > Angelo Nicolosi wrote:
> > > It's possible that the problem is in my C code but every time that
> I'm
> > > allocating memory, using always the palloc() function, I'm always
> > > calling the pfree().
> > > There is some way to analyze the code meanwhile is working inside
> the
> > > Postgre server (something like valgrind)?
> > > However the command free -m on my machine outputs:
> > >
> > > total used free shared buffers cached
> > > Mem: 2010 664 1345 0 157 383
> > > -/+ buffers/cache: 123 1886
> > > Swap: 16386 41 16345
> > >
> > > I think that the swap is enough.
> > > Could you give me some tips about how can I see where is the
> problem?
> > > Thank you for your help!
> >
> > do you know what query you were making when you ran out of memory?
> it
> > -appears- it was a postgres server process that was OOM'd.
> >
> > what OS and version are you on (OOM seems to imply its likely
> linux,
> > since no other OS I'm familiar with would randomly kill processes
> like
> > that), what version postgres, etc ?
> >
> > also, what are the various memory settings in your postgresql.conf
> > (shared_buffers, work_mem, etc)
> >
> >
>
>
>
> ______________________________________________________________________
> Una risposta istantanea? Usa Messenger da Hotmail
--
Hannu Krosing   http://www.2ndQuadrant.com
PostgreSQL Scalability and Availability
   Services, Consulting and Training