On Thu, May 11, 2006 at 09:50:14PM +0200, Martijn van Oosterhout wrote:
> On Thu, May 11, 2006 at 02:39:14PM -0500, Jim C. Nasby wrote:
> > On Thu, May 11, 2006 at 07:39:23PM +0200, Martijn van Oosterhout wrote:
> > > This is an idle backend waiting for the user.
> >
> > So why would that be waiting to lock the socket? My understanding is
> > that nothing else should be contending for that socket, no?
>
> I'm not sure about locks, but it will be blocking on the socket...
Yeah, talking to AndrewSN and some others on IRC they're wondering if
maybe it's running out of network buffers, which could possibly cause
this.
> > > > #0 0x000000080137638c in sendto () from /lib/libc.so.6
> > > > #1 0x0000000000535e67 in pgstat_report_tabstat () at pgstat.c:846
> > >
> > > This definitly the statistics collector, which is something that was
> > > speculated upthread. Do you get a lot of these?
> >
> > I included everything that was captured, but of course that's only a
> > small sampling. If it's helpful we could probably setup something that
> > would automatically grab stack traces for a larger number of backends
> > and then see how many were in that state.
>
> If you know the pids you should be able to within gdb just do
> attach/bt/detech. gdb has some redimentary scripting capabilites so you
> might be able to do this fairly quickly.
Yeah, what I was thinking. But now that we've been investigating this
more I'm suspecting this is more a matter of tuning over it being a
possible bug. It turns out we have to be doing over 2000 queries per
second on this dual opteron before the problem will happen...
> > Yeah, my suspicion is that those processes had moved past waiting on the
> > socket lock by the time gdb got to them. Any idea of how you could tell
> > what state (as reported by top) the process was in when gdb stopped it?
>
> Heh. Attaching to a process has the same effect as sending it a signal.
> Any active system call is aborted and gdb traps it as it goes to
> userspace. So by definition it's in running state when gdb gets it ...
Heh, yeah... oh well.
--
Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com
Pervasive Software http://pervasive.com work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461