While working on removing FP macros of the geometric type operators,
I noticed that line intersection is giving wrong results:
regression=# select '{1,1,1}'::line # '{2,1,0}'::line;
?column?
----------
(1,2)
(1 row)
There are a few other operators relying on this function which are also
returning wrong results. Segfaults should be possible at least on
assert enabled builds, though FP macros makes it hard to get them.
The bug seems to be introduced by 215bc83d on 1997. I couldn't find
any relevant bug report on the archives except Tom Lane's doubtful
comment [1] from 1999. Patch attached to fix the formula.
[1] https://www.postgresql.org/message-id/5537.941498236%40sss.pgh.pa.us