Обсуждение: More idle thoughts

The Linux kernel had a big push to reduce latency, and one of the
tricks they did was they replaced the usual interrupt points with a
call which noted how long it had been since the last interrupt point.
It occurs to me we could do the same for CHECK_FOR_INTERRUPTS() by
conditionally having it call a function which calls gettimeofday and
compares with the previous timestamp received at the last CFI().

I think the only gotcha would be at CHECK_FOR_INTERRUPTS() calls which
occur after a syscall that an actual interrupt would interrupt. They
might show a long wait even though an actual interrupt would trigger
them right away --  the two that come to mind are reading from the
client and blocking on a semaphore. So I think we would need to add a
parameter to indicate if that's the case for a given call-site. I
think it would be easy to know what value to put there because I think
we always do a CFI() immediately after such syscalls but I'm not 100%
sure that's the case.

Obviously this wouldn't run all the time. I'm not even sure it should
run on the build farm because I think doing an extra syscall at these
places might mask timing bugs by synchronizing the bus in a lot of
places. But even a few build farm animals might uncover places where
we don't respond to C-c or hold up the sinval messages etc.

It also doesn't replace our current method of responding to user
complaints -- many if not all of them are relatively subtle cases
where the user is doing something unusual to create a loop that
doesn't normally occur. But we won't know that unless we try.

-- 
greg

Greg Stark <stark@mit.edu> writes:
> The Linux kernel had a big push to reduce latency, and one of the
> tricks they did was they replaced the usual interrupt points with a
> call which noted how long it had been since the last interrupt point.
> It occurs to me we could do the same for CHECK_FOR_INTERRUPTS() by
> conditionally having it call a function which calls gettimeofday and
> compares with the previous timestamp received at the last CFI().

Hmm.  The only thing you could find out is which CFI call was reporting
the long delay, not what the path of control in between had been.  If
the long computation had been down inside some function that's no longer
active, this wouldn't help much to track it down.  Still, it'd be better
than no data at all.

> I think the only gotcha would be at CHECK_FOR_INTERRUPTS() calls which
> occur after a syscall that an actual interrupt would interrupt. They
> might show a long wait even though an actual interrupt would trigger
> them right away --  the two that come to mind are reading from the
> client and blocking on a semaphore. So I think we would need to add a
> parameter to indicate if that's the case for a given call-site. I
> think it would be easy to know what value to put there because I think
> we always do a CFI() immediately after such syscalls but I'm not 100%
> sure that's the case.

I'm afraid you're being too optimistic about that.  What I'd think about
is just adding an extra call to reset the delay-time counter after each
such syscall, rather than having two kinds of CFI.

> Obviously this wouldn't run all the time.

Yeah, it would increase the overhead of CFI by orders of magnitude,
so you wouldn't even want to think about building a production version
that way.  But it might be a useful testing option.
        regards, tom lane

On Fri, 2010-03-26 at 18:59 +0000, Greg Stark wrote:

> The Linux kernel had a big push to reduce latency, and one of the
> tricks they did was they replaced the usual interrupt points with a
> call which noted how long it had been since the last interrupt point.
> It occurs to me we could do the same for CHECK_FOR_INTERRUPTS() by
> conditionally having it call a function which calls gettimeofday and
> compares with the previous timestamp received at the last CFI().

Reducing latency sounds good, but what has CFI got to do with that?

-- Simon Riggs           www.2ndQuadrant.com

Simon Riggs <simon@2ndQuadrant.com> writes:
> On Fri, 2010-03-26 at 18:59 +0000, Greg Stark wrote:
>> It occurs to me we could do the same for CHECK_FOR_INTERRUPTS() by
>> conditionally having it call a function which calls gettimeofday and
>> compares with the previous timestamp received at the last CFI().

> Reducing latency sounds good, but what has CFI got to do with that?

It took me about five minutes to figure out what Greg was on about too.
His point is that we need to locate code paths in which an extremely
long time can pass between successive CFI calls, because that means
the backend will fail to respond to SIGINT/SIGTERM for a long time.
Instrumenting CFI itself is a possible tool for that.
        regards, tom lane

On Sun, 2010-03-28 at 12:47 -0400, Tom Lane wrote:
> Simon Riggs <simon@2ndQuadrant.com> writes:
> > On Fri, 2010-03-26 at 18:59 +0000, Greg Stark wrote:
> >> It occurs to me we could do the same for CHECK_FOR_INTERRUPTS() by
> >> conditionally having it call a function which calls gettimeofday and
> >> compares with the previous timestamp received at the last CFI().
> 
> > Reducing latency sounds good, but what has CFI got to do with that?
> 
> It took me about five minutes to figure out what Greg was on about too.
> His point is that we need to locate code paths in which an extremely
> long time can pass between successive CFI calls, because that means
> the backend will fail to respond to SIGINT/SIGTERM for a long time.
> Instrumenting CFI itself is a possible tool for that.

I was thinking we could do this via signals, but actually instrumenting
the code paths seems better.

There probably are a few paths still to improve. Dare I suggest we
follow the tried and tested open source approach of wait-for-complaint?

Reducing latency elsewhere would be time better spent (!). I was
thinking of adding a "reason" field onto ReadBuffer, so we can diagnose
the source of buffer waits.

-- Simon Riggs           www.2ndQuadrant.com