Re: pg_ctl reload breaks our client

Michael,
Many thanks for your response; it is much appreciated.  My responses are
embedded below:

On Fri, 2005-09-16 at 17:10 -0600, Michael Fuhr wrote:
> On Fri, Sep 16, 2005 at 02:16:29PM -0700, Marc Munro wrote:
> > It is Postgres 7.3.6.  The client is a multi-threaded C++ client.  The
> > breakage was that one group of connections simply stopped.  Others
> > contined without problem.  It is not clear exactly what was going on.
>
> How did the connections "stop"?  Were the connections broken, causing
> queries to fail?  Or did queries block and never return?  Or something
> else?  What was happening that shouldn't happen, or what wasn't
> happening that should happen?

From the server side, there were simply connections (1 or 2) that
appeared idle.  From the client side it looked like a query had been
initiated but the client thread was stuck in a library call (as near as
we can tell).  This, vague though it is, is as much as I know right now.
We were unable to do much debugging as it is a production system and the
priority was to get it back up.

> If the connections were still active but not returning, did you do
> a process trace on the connection's postmaster or attach a debugger
> to it to see what it was doing?

No, time pressure prevented this.

> Could the timing of the problem have been coincidence?  Have you
> ever seen the problem without a reload?  How often do you see the
> problem after a reload?  Do you know for certain that the application
> was working immediately before the reload and not working immediately
> after it?

It *could* be coincidence, but the problem began within 5 seconds of the
reload.  Coincidence is unlikely.

> What operating system are you using?

Linux 2.4.20 smp i686

>
> > Nothing in our application logs gives us any clue to this.
>
> What about the postmaster logs?

Ah, now there's another story.  Unavailable I'm afraid.  Resolving that
is also on my priority list.

> > As for reproducibility, it has hapenned before in test environments when
> > we have bounced the datanase.  This is not too shocking as I would
> > expect the client to notice this :-)  It is a little more shocking when
> > it's a reload.  Or maybe I have simply misunderstood what reload does.
>
> Can you reproduce the problem with a reload?  A stop and start will
> terminate client connections, but a reload shouldn't.

This is not currently seen as a priority (the work-around of "don't do
that" is seen as sufficient).  I'm simply hoping to get someone to say
for sure that the client app should not be able to tell that a reload
has happened.  At that point I may be able to raise the priority of this
issue.

I would certainly like to do more investigation.  If postgresql hackers
are interested in this strange event (please tell me for sure that it
*is* strange) that may also help me to get the necessary resources to
run more tests.

Thanks again.

__
Marc Munro