On 2013-12-12 21:15:29 -0500, Tom Lane wrote:
> Christophe Pettus <xof@thebuild.com> writes:
> > On Dec 12, 2013, at 5:45 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >> Presumably, we are seeing the victim rather than the perpetrator of
> >> whatever is going wrong.
>
> > This is probing about a bit blindly, but the only thing I can see about this system that is in some way unique (and
thisis happening on multiple machines, so it's unlikely to be hardware) is that there are a relatively large number of
relations(like, 440,000+) distributed over many schemas. Is there anything that pins a buffer that is O(N) to the
numberof relations?
>
> It's not a buffer *pin* that's at issue, it's a buffer header spinlock.
> And there are no loops, of any sort, that are executed while holding
> such a spinlock. At least not in the core PG code. Are you possibly
> using any nonstandard extensions?
It could maybe be explained by a buffer aborting while performing
IO. Until it has call AbortBufferIO(), other backends will happily loop
in WaitIO(), constantly taking the the buffer header spinlock and
locking io_in_progress_lock in shared mode, thereby preventing
AbortBufferIO() from succeeding.
Christophe: are there any "unusual" ERROR messages preceding the crash,
possibly some minutes before?
Greetings,
Andres Freund
-- Andres Freund http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services