Обсуждение: Memory for BYTEA returned by C function is not released until connection is dropped

Поиск
Список
Период
Сортировка

Memory for BYTEA returned by C function is not released until connection is dropped

От
John Leiseboer
Дата:
I have written a number of functions in C that return BYTEA type. I have compiled and run on both Windows and Linux,
32-bitand 64-bit, PostgreSQL versions 9.3 and 9.4. 

My functions return BYTEA data to the caller. The problem is that memory usage grows until there is no memory left on
thehost, at which point an error is returned. If I drop the connection (e.g. by quitting from pqsql), the memory is
returned.

I wrote the following minimal function to test palloc() and BYTEA return behaviour, and found that this minimal program
alsoexhibits the unbounded memory growth problem. 


C source code:

PG_FUNCTION_INFO_V1(test_palloc);
Datum test_palloc()
{
    bytea *test_ret;
    int test_len = 1024;

    test_ret = (bytea *)palloc(test_len + VARHDRSZ);
    SET_VARSIZE(test_ret, test_len + VARHDRSZ);
    PG_RETURN_BYTEA_P(test_ret);
}

Function definition:

CREATE OR REPLACE FUNCTION test_palloc() RETURNS BYTEA
AS E'<path to shared library>', test_palloc' LANGUAGE C IMMUTABLE STRICT;

psql commands to reproduce the problem:

\o out.txt
SELECT ids.*, test_palloc() FROM GENERATE_SERIES(1, 1000000) ids;

At the completion of the above command, host memory will have been consumed but not released back to the system. After
quittingpsql (\q), memory is released. 

Is this expected behaviour or a bug? Am I doing something wrong? How can I return a BYTEA type from a C library
functionwithout having to drop the connection in order to recover the allocated memory that is returned to the caller? 

Regards,
John


Re: Memory for BYTEA returned by C function is not released until connection is dropped

От
Pavel Stehule
Дата:


2015-09-21 4:31 GMT+02:00 John Leiseboer <JL@quintessencelabs.com>:
I have written a number of functions in C that return BYTEA type. I have compiled and run on both Windows and Linux, 32-bit and 64-bit, PostgreSQL versions 9.3 and 9.4.

My functions return BYTEA data to the caller. The problem is that memory usage grows until there is no memory left on the host, at which point an error is returned. If I drop the connection (e.g. by quitting from pqsql), the memory is returned.

I wrote the following minimal function to test palloc() and BYTEA return behaviour, and found that this minimal program also exhibits the unbounded memory growth problem.


C source code:

PG_FUNCTION_INFO_V1(test_palloc);
Datum test_palloc()
{
        bytea *test_ret;
        int test_len = 1024;

        test_ret = (bytea *)palloc(test_len + VARHDRSZ);
        SET_VARSIZE(test_ret, test_len + VARHDRSZ);
        PG_RETURN_BYTEA_P(test_ret);
}

Function definition:

CREATE OR REPLACE FUNCTION test_palloc() RETURNS BYTEA
AS E'<path to shared library>', test_palloc' LANGUAGE C IMMUTABLE STRICT;

psql commands to reproduce the problem:

\o out.txt
SELECT ids.*, test_palloc() FROM GENERATE_SERIES(1, 1000000) ids;

At the completion of the above command, host memory will have been consumed but not released back to the system. After quitting psql (\q), memory is released.

Is this expected behaviour or a bug? Am I doing something wrong? How can I return a BYTEA type from a C library function without having to drop the connection in order to recover the allocated memory that is returned to the caller?


This memory is release, when memory context is dropped.

http://www.neilconway.org/talks/hacking/hack_slides.pdf look on slide 15

Regards

Pavel
 
Regards,
John


--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general