Re: excessive amounts of consumed memory (RSS), triggering OOM killer

On 2.12.2014 00:31, Andrew Dunstan wrote:
> 
> On 12/01/2014 05:39 PM, Tomas Vondra wrote:
>> Hi all,
>>
>> while working on the patch decreasing amount of memory consumed by
>> array_agg [1], I've ran into some strange OOM issues. Reproducing them
>> using the attached SQL script is rather simple.
>>
>>    [1] https://commitfest.postgresql.org/action/patch_view?id=1652
>>
>> At first I thought there's some rare hidden memory leak, but I'm pretty
>> sure that's not the case. I've even put some explicit counters into
>> aset.c counting allocated/freed blocks, and it seems to be working just
>> fine (and matching the context tree perfectly). So no memory leak.
> 
> 
> Doesn't this line:
> 
>    TopMemoryContext: 136614192 total in 16678 blocks; 136005936 free
> (500017 chunks); 608256 used
> 
> 
> look pretty suspicious?

It certainly looks a bit large, especially considering this is a fresh
cluster with a single DB, containing a single table (created by user),
no doubt about that.

For comparison, this is a new backend:

TopMemoryContext: 78128 total in 11 blocks; 8528 free (16 chunks); 69600
used

OTOH, it's "just" 130MB, and the RSS values are ~6GB when the OOM hits.


For the record, I only tweaked these settings in the config file:

test=# select name, setting from pg_settings where source like
'%configuration file%';           name            |      setting
----------------------------+-------------------checkpoint_segments        | 32DateStyle                  | ISO,
DMYdefault_text_search_config| pg_catalog.simpledynamic_shared_memory_type | posixlc_messages                |
cs_CZ.UTF-8lc_monetary               | cs_CZ.UTF-8lc_numeric                 | cs_CZ.UTF-8lc_time                    |
cs_CZ.UTF-8log_timezone              | Europe/Praguemaintenance_work_mem       | 524288max_connections            |
100shared_buffers            | 65536TimeZone                   | Europe/Praguework_mem                   | 1048576
 
(14 rows)

Some of those are set by the initdb, of course.

regards
Tomas