"Nick Roosevelt" <nroose@thepinc.com> writes:
> I am getting bad results for distance between point and lseg. As you can
> see below, the first result is correct, and the second is clearly not.
Hmm ... what it looks like to me is that there's an ancient thinko
in dist_ps_internal(). It's trying to calculate the slope of the
perpendicular to the given line segment, and it gets it wrong.
The segment's own slope would be deltaY / deltaX, so the slope
of the perpendicular should be the negative inverse of that, ie
-deltaX / deltaY, but what it was actually calculating was
-deltaY / deltaX. So it was getting the wrong answers for any
situation where the given line segment's slope wasn't +1/-1
(or 0 or infinite, which are correctly special-cased).
Depressingly, fixing this changes none of the regression test outputs;
apparently all the test cases involving distances were one of the
special cases. But it's really amazing no one complained of this
before ...
regards, tom lane