OK, indeed this is a bug --- the planner has a bad interaction for LIMIT
clauses used in "IN" subselects. Will fix.
You could avoid the bug, and get a better plan at the top level anyway,
by writing
SELECT frm,rcpt,subject FROM mesg_headers WHERE mesgnum = (SELECT mesgnum
FROM mesg_headers ORDER BY mesgnum DESC LIMIT 1);
Since you know the subselect is going to produce exactly one tuple,
there's no need to use IN.
regards, tom lane