Trevor Talbot wrote:
> On 10/21/07, Magnus Hagander <magnus@hagander.net> wrote:
>
>>> I tried generating idle connections in an effort to reproduce
>>> Laurent's problem, but I ran into a local limit instead: for each
>>> backend, postmaster creates a thread and burns 4MB of its 2GB address
>>> space. It fails around 490.
>> Oh, that's interesting. That's actually a sideeffect of us increasing
>> the stack size for the postgres.exe executable in order to work on other
>> things. By default, it burns 1MB/thread, but ours will do 4MB. Never
>> really thought of the problem that it'll run out of address space.
>> Unfortunately, that size can't be changed in the CreateThread() call -
>> only the initially committed size can be changed there.
>>
>> There are two ways to get around it - one is not using a thread for each
>> backend, but a single thread that handles them all and then some sync
>> objects around it. We originally considered this but said we won't
>> bother changing it because the current way is simpler, and the overhead
>> of a thread is tiny compared to a process. I don't think anybody even
>> thought about the fact that it'd run you out of address space...
>
> I'd probably take the approach of combining win32_waitpid() and
> threads. You'd end up with 1 thread per 64 backends; when something
> interesting happens the thread could push the info onto a queue, which
> the new win32_waitpid() would check. Use APCs to add new backends to
> threads with free slots.
I was planning to make it even easier and let Windows do the job for us,
just using RegisterWaitForSingleObject(). Does the same - one thread per
64 backends, but we don't have to deal with the queueing ourselves.
Should be rather trivial to do.
Keeps win32_waitpid() unchanged.
That said, refactoring win32_waitpid() to be based on a queue might be a
good idea *anyway*. Have the callback from above put something in the
queue, and go with your idea for the rest.
//Magnus