Attached revision:
* Still doesn't address the open question of whether or not we should
optimistically always try "memcmp() == 0" on tiebreak. I still lean
towards "yes".
* Leaves open the question of what to do when we can't use the
abbreviated keys optimization just because a datum tuplesort is
preferred when sorting single-attribute tuples (recall that datum case
tuplesorts cannot use abbreviated keys). We want to avail of tuplesort
datum sorting where we can, except when abbreviated keys are
available, which presumably tips the balance in favor of heap tuple
sorting even when sorting on only one attribute, simply because then
we can then use abbreviated keys. I'm thinking in particular of
nodeAgg.c, which is an important case.
There are still FIXME/TODO comments for each of these two points.
Further, this revised/rebased patch set:
* Incorporates your feedback on stylistic issues, with changes
confined to their own commit (on top of earlier commits that are
almost, but not quite, the same as the prior revision that your
remarks apply to).
* No longer does anything special within reversedirection_heap(),
since that is unnecessary, as it's only used by bounded sorts, which
aren't a useful target for abbreviated keys. This is noted. There is
no convenient point to add a defensive assertion against this, so I
haven't.
* Updates comments in master in a broken-out way, reflecting opclass
contract with sortsupport as established by
1d41739e5a04b0e93304d24d864b6bfa3efc45f2, that is convenient to apply
to and commit in the master branch immediately.
--
Peter Geoghegan