Re: libpq and multi-threading

That is not a thread. Linux man clone right at the start …

“clone, __clone2, clone3 - create a child process”

What you want is pthread_create (or similar)

There’s a bunch of not well documented dragons if you’re trying to treat a child process as a thread. Use POSIX Threads, as pretty much anytime PG or anything else Linux based says thread they’re talking about a POSIX Thread environment.

On Wed, May 3, 2023 at 05:12 Michael J. Baars <mjbaars1977.pgsql.hackers@gmail.com> wrote:

Hi Peter,

The shared common address space is controlled by the clone(2) CLONE_VM option. Indeed this results in an environment in which both the parent and the child can read / write each other's memory, but dynamic memory being allocated using malloc(3) from two different threads simulaneously will result in internal interference. 

Because libpq makes use of malloc to store results, you will come to find that the CLONE_VM option was not the option you were looking for.

On Tue, 2 May 2023, 19:58 Peter J. Holzer, <hjp-pgsql@hjp.at> wrote:

On 2023-05-02 17:43:06 +0200, Michael J. Baars wrote:
> I don't think it is, but let me shed some more light on it.

One possibly quite important information you haven't told us yet is
which OS you use.

Or how you create the threads, how you pass the results around, what
else you are possibly doing between getting the result and trying to use
it ...

A short self-contained test case might shed some light on this.


> After playing around a little with threads and memory, I now know that the
> PGresult is not read-only, it is read-once. The child can only read that
> portion of parent memory, that was written before the thread started. Read-only
> is not strong enough.
>
> Let me correct my first mail. Making libpq use mmap is not good enough either.
> Shared memory allocated by the child can not be accessed by the parent.

Are you sure you are talking about threads and not processes? In the OSs
I am familiar with, threads (of the same process) share a common address
space. You don't need explicit shared memory and there is no such thing
as "parent memory" (there is thread-local storage, but that's more a
compiler/library construct).

        hp

--
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp@hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"

--

"Genius might be described as a supreme capacity for getting its possessors
into trouble of all kinds."
-- Samuel Butler