Обсуждение: C-procedure crashed in Postgres 8.3.3 when using 'text' variable (WinXP)

Hello!
I'd like to write C-function returning text for using in PG 8.3.3. (WinXP SP2)
For compilation I use MinGW-5.1.4 (gcc 3.4.5),MSYS-1.0.10.
The code looks like this (in reduced variant):

#include "postgres.h"
#include "fmgr.h"
#include "executor/executor.h"
#include "utils/timestamp.h"
#include "utils/builtins.h"
#include "utils/formatting.h"

#ifdef PG_MODULE_MAGIC
PG_MODULE_MAGIC;
#endif

#define PG_CSTR_GET_TEXT(cstrp) \
DatumGetTextP(DirectFunctionCall1(textin, CStringGetDatum(cstrp)))

PG_FUNCTION_INFO_V1(SomeFunction);
Datum
SomeFunction(PG_FUNCTION_ARGS)
{
   PG_RETURN_TEXT_P(PG_CSTR_GET_TEXT("my_string"));
}

Then I create stored procedure in PG:

CREATE OR REPLACE FUNCTION "service"."some_function" () RETURNS text AS
     '$libdir/some_dll', 'SomeFunction'
     LANGUAGE C STRICT;

-----------------------------------------------------
When I'm trying to use "service"."some_function" PG fails.

I tried to use the example from documentation:
PG_FUNCTION_INFO_V1(concat_text);

Datum
concat_text(PG_FUNCTION_ARGS)
{
    text  *arg1 = PG_GETARG_TEXT_P(0);
    text  *arg2 = PG_GETARG_TEXT_P(1);
    int32 new_text_size = VARSIZE(arg1) + VARSIZE(arg2) - VARHDRSZ;
    text *new_text = (text *) palloc(new_text_size);

    SET_VARSIZE(new_text, new_text_size);
    memcpy(VARDATA(new_text), VARDATA(arg1), VARSIZE(arg1) - VARHDRSZ);
    memcpy(VARDATA(new_text) + (VARSIZE(arg1) - VARHDRSZ),
           VARDATA(arg2), VARSIZE(arg2) - VARHDRSZ);
    PG_RETURN_TEXT_P(new_text);
}

CREATE FUNCTION concat_text(text, text) RETURNS text
     AS '$libdir/some_dll', 'concat_text'
     LANGUAGE C STRICT;

And I got the same result.

When I'm trying to return integer, f.e., evetything is fine.
Also all this examples worked fine with PostgreSQL 8.2.


I saw some similar problems - but I didn't find the solution (
http://archives.postgresql.org/pgsql-general/2008-05/msg00060.php
http://archives.postgresql.org/pgsql-general/2008-05/msg00097.php

Have somebody any ideas how to resolve this problem?

Thanks in advance,
Marina

el dorado <do_ra_do@mail.ru> writes:
> I'd like to write C-function returning text for using in PG 8.3.3. (WinXP SP2)
> For compilation I use MinGW-5.1.4 (gcc 3.4.5),MSYS-1.0.10.

Was the Postgres server you're using built the same way?  I seem to
recall some incompatibilities between MinGW and MSVC builds.

            regards, tom lane

Tom Lane wrote:
> el dorado <do_ra_do@mail.ru> writes:
>> I'd like to write C-function returning text for using in PG 8.3.3. (WinXP SP2)
>> For compilation I use MinGW-5.1.4 (gcc 3.4.5),MSYS-1.0.10.
>
> Was the Postgres server you're using built the same way?  I seem to
> recall some incompatibilities between MinGW and MSVC builds.

I'd be surprised if there weren't some given how tricky it is to mix
different compilers on Windows, where that usually means mixing C
runtimes as well. On any platform it's best to stick to the same C
runtime for all libraries to be linked into the same process*.

That said, PostgreSQL does appear to use its own allocator for memory
passed across library boundaries, and if it doesn't pass any FILE
pointers across library boundaries either then that's the worst two
problems that arise with a C runtime mismatch taken care of. As far as I
know, anyway, and I've had the misfortune to have to dig into it a bit.

It seems like a good idea to use the same version of the same compiler
that your version of PostgreSQL was built with anyway. If nothing else,
I've only taken a cursory look at the code and have little knowledge
about it, so there could well be code paths/uses where memory is
malloc()'d in one DLL and freed()'d in another, etc. I'm sure there all
sorts of other possible wrinkles, and it'd just be easier to stick with
the same compiler. You can download Visual Studio 2005 Express Edition
for free from MS.

* Thankfully UNIXes tend to be sane enough to use the same C library
across all C compilers installed (and generally stick to one compiler
anyway), so most of us are able to avoid this exciting little complication.

--
Craig Ringer

Craig Ringer wrote:

> That said, PostgreSQL does appear to use its own allocator for memory
> passed across library boundaries, and if it doesn't pass any FILE
> pointers across library boundaries either then that's the worst two
> problems that arise with a C runtime mismatch taken care of. As far as I
> know, anyway, and I've had the misfortune to have to dig into it a bit.

Note that our allocator uses malloc() underneath.  And I would think
that we do pass FILE pointers around too; AllocateFile works with FILE,
so any library that's using that method to get to files would be in
trouble (which they should, because it's the way to ensure we don't leak
files on transaction abort and also to ensure that the system continues
to work on the face of lots of open files).

--
Alvaro Herrera                                http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Alvaro Herrera wrote:
> Craig Ringer wrote:
>
>> That said, PostgreSQL does appear to use its own allocator for memory
>> passed across library boundaries, and if it doesn't pass any FILE
>> pointers across library boundaries either then that's the worst two
>> problems that arise with a C runtime mismatch taken care of. As far as I
>> know, anyway, and I've had the misfortune to have to dig into it a bit.
>
> Note that our allocator uses malloc() underneath.

Sure. That's fine, so long as the wrapper functions palloc and pfree are
real functions that reside in the same DLL on windows. You can pass
pointers around freely, otherwise you couldn't do much at all ... the
problem is with a call path like this:

- DLL2 calls a function in DLL1
- DLL1 malloc()'s some memory and returns a pointer to it
- DLL2 free()'s the pointer

My understanding of the reason that this breaks if DLL1 and DLL2 are
linked to different C runtimes is that it's essentially a double free
plus a memory leak. The allocator in DLL2 has never allocated the memory
so the free call is freeing unallocated memory. It's never properly
freed by the allocator in DLL2, so it's leaked as far as that allocator
is concerned. It's quite possible that my understanding of the problem
isn't correct, though.

In any case, the safe thing to do is:

- DLL2 calls a function in DLL1
- DLL1 malloc()'s some memory and returns a pointer to it
- DLL2 calls a free() wrapper function defined in DLL1
- The free() wrapper in DLL1 calls the real free() to release the memory

If palloc() and pfree() are real functions defined in the same DLL, then
it should be just fine as use of them will ensure that everything uses
the same underlying memory allocator. A problem might arise if Pg ever
passes memory not allocated with palloc() around, though.

It looks like palloc is a macro for the real function
MemoryContextAlloc, and pfree is just a real function - so it should be
fine. They don't expose references to malloc() and free() directly.

This actually applies to any situation in which multiple malloc()/free()
implementations are used - but most people and platforms are sane enough
to avoid that.

> And I would think
> that we do pass FILE pointers around too; AllocateFile works with FILE,
> so any library that's using that method to get to files would be in
> trouble (which they should, because it's the way to ensure we don't leak
> files on transaction abort and also to ensure that the system continues
> to work on the face of lots of open files).

Yep, I'm pretty sure that'll cause issues when using a mingw-compiled
object in a msvc++-compiled copy of Pg.

--
Craig Ringer