Обсуждение: Valgrind mem-check for postgres extension

Поиск
Список
Период
Сортировка

Valgrind mem-check for postgres extension

От
Natarajan R
Дата:
Hi pg-hackers,

I have written a postgres extension, and know that memory leak check can be done with valgrind. With the help of postgres_valgrind_wiki started postgres server with the valgrind(as given in the wiki)
valgrind --leak-check=no --gen-suppressions=all \    --suppressions=src/tools/valgrind.supp --time-stamp=yes \    --error-markers=VALGRINDERROR-BEGIN,VALGRIND ERROR-END \    --log-file=$HOME/pg-valgrind/%p.log --trace-children=yes \    postgres --log_line_prefix="%m %p " \    --log_statement=all --shared_buffers=64MB 2>&1 | tee $HOME/pg-valgrind/postmaster.log
I have few doubts in here,

1. When I run with --leak-check=full, I get memory leaks for postgres functions under possibly or definitely lost categories.. Is this expected? If yes, how shall i ignore it?(by creating .supp?).. kindly suggest
2. Is there any other way to test my extension memory leaks alone, because combining with postgres leaks is making instrumentation complex?..
3. I have seen some macros for valgrind support within postgres source code under utils/memdebug.h, but couldn't get complete idea of using it from the comments in pg_config_manual.h under USE_VALGRIND macro, pls provide some guidance here..

Thank you,
Natarajan R

Re: Valgrind mem-check for postgres extension

От
Tom Lane
Дата:
Natarajan R <nataraj3098@gmail.com> writes:
> I have few doubts in here,

> 1. When I run with *--leak-check=full*, I get memory leaks for postgres
> functions under possibly or definitely lost categories.. Is this expected?

Maybe ... you did not show your test case, so it's hard to say.  But it
could well be that this is an artifact of failing to define USE_VALGRIND.

> 2. Is there any other way to test my extension memory leaks alone, because
> combining with postgres leaks is making instrumentation complex?..

No, not really.

> 3. I have seen some macros for valgrind support within postgres source code
> under utils/memdebug.h, but couldn't get complete idea of using it from the
> comments in pg_config_manual.h under *USE_VALGRIND *macro, pls provide some
> guidance here..

If you didn't build the core code with USE_VALGRIND defined, then none of
this stuff is going to work ideally.

The way I like to do it is to run configure, and then manually add
"#define USE_VALGRIND" to the generated src/include/pg_config.h
file before invoking "make".  Probably other people have different
habits.

            regards, tom lane

Re: Valgrind mem-check for postgres extension

От
Andrew Dunstan
Дата:
On 2022-05-18 We 01:12, Tom Lane wrote:
> Natarajan R <nataraj3098@gmail.com> writes:
>> I have few doubts in here,
>> 1. When I run with *--leak-check=full*, I get memory leaks for postgres
>> functions under possibly or definitely lost categories.. Is this expected?
> Maybe ... you did not show your test case, so it's hard to say.  But it
> could well be that this is an artifact of failing to define USE_VALGRIND.
>
>> 2. Is there any other way to test my extension memory leaks alone, because
>> combining with postgres leaks is making instrumentation complex?..
> No, not really.
>
>> 3. I have seen some macros for valgrind support within postgres source code
>> under utils/memdebug.h, but couldn't get complete idea of using it from the
>> comments in pg_config_manual.h under *USE_VALGRIND *macro, pls provide some
>> guidance here..
> If you didn't build the core code with USE_VALGRIND defined, then none of
> this stuff is going to work ideally.
>
> The way I like to do it is to run configure, and then manually add
> "#define USE_VALGRIND" to the generated src/include/pg_config.h
> file before invoking "make".  Probably other people have different
> habits.


The standard buildfarm config uses these for valgrind builds:

        CFLAGS   => "-fno-omit-frame-pointer -O0 -fPIC",
        CPPFLAGS => "-DUSE_VALGRIND  -DRELCACHE_FORCE_RELEASE",

cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

Re: Valgrind mem-check for postgres extension

От
Natarajan R
Дата:
>> I have few doubts in here,
>> 1. When I run with *--leak-check=full*, I get memory leaks for postgres
>> functions under possibly or definitely lost categories.. Is this expected?

> Maybe ... you did not show your test case, so it's hard to say.  But it
> could well be that this is an artifact of failing to define USE_VALGRIND

I did not run any test, just simply started the plain postgres(my extension not installed) with valgrind(command given below)..
valgrind --leak-check=full --suppressions=src/tools/valgrind.supp --time-stamp=yes --error-markers=VALGRINDERROR-BEGIN,VALGRINDERROR-END --trace-children=yes --log-file=pg_valgrind/pg-valgrind_%p.log postgres -D data

>> 3. I have seen some macros for valgrind support within postgres source code
>> under utils/memdebug.h, but couldn't get complete idea of using it from the
>> comments in pg_config_manual.h under *USE_VALGRIND *macro, pls provide some
>> guidance here..

> If you didn't build the core code with USE_VALGRIND defined, then none of
> this stuff is going to work ideally.

> The way I like to do it is to run configure, and then manually add
> "#define USE_VALGRIND" to the generated src/include/pg_config.h
> file before invoking "make".  Probably other people have different
> habits.

I tried the things like you said 
1. Pg configured already, so #define USE_VALGRIND in src/include/pg_config.h
2. make, make install
3. started pg with valgrind(same command mentioned above).. 
Now, memory leaks are being reported for palloc, malloc, MemoryContextAlloc functions..(sample image from a valgrind report for a process attached).
I couldn't specify from which process, this was generated.. as the valgrind provide pid alone..

image.png

In an overview, I am trying to test a memory leak for my extension, correct me if i am doing it the wrong way..

Regards,
Natarajan R


On Wed, 18 May 2022 at 21:38, Andrew Dunstan <andrew@dunslane.net> wrote:

On 2022-05-18 We 01:12, Tom Lane wrote:
> Natarajan R <nataraj3098@gmail.com> writes:
>> I have few doubts in here,
>> 1. When I run with *--leak-check=full*, I get memory leaks for postgres
>> functions under possibly or definitely lost categories.. Is this expected?
> Maybe ... you did not show your test case, so it's hard to say.  But it
> could well be that this is an artifact of failing to define USE_VALGRIND.
>
>> 2. Is there any other way to test my extension memory leaks alone, because
>> combining with postgres leaks is making instrumentation complex?..
> No, not really.
>
>> 3. I have seen some macros for valgrind support within postgres source code
>> under utils/memdebug.h, but couldn't get complete idea of using it from the
>> comments in pg_config_manual.h under *USE_VALGRIND *macro, pls provide some
>> guidance here..
> If you didn't build the core code with USE_VALGRIND defined, then none of
> this stuff is going to work ideally.
>
> The way I like to do it is to run configure, and then manually add
> "#define USE_VALGRIND" to the generated src/include/pg_config.h
> file before invoking "make".  Probably other people have different
> habits.


The standard buildfarm config uses these for valgrind builds:

        CFLAGS   => "-fno-omit-frame-pointer -O0 -fPIC",
        CPPFLAGS => "-DUSE_VALGRIND  -DRELCACHE_FORCE_RELEASE",

cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

Вложения