Also ...
"Robert Bedell" <robert@friendlygenius.com> writes:
> ... I think that LookupTupleHashEntry
> is the best place to put the code to spill over unto disk, since that is
> used only by the Group, Agg, and Subplan executor nodes.
That's the right area to be looking at, but I don't think you can expect
to do a decent job with localized hacking in LookupTupleHashEntry. That
function's API is predicated on the assumption that you have random
access to any entry in the hash table --- which stops being true as soon
as you spill some entries to disk. The key problem here is to figure
out how to reorganize the computation to cope efficiently with nonrandom
access to the hash table. You might want to look at the algorithm
embodied in nodeHash.c/nodeHashjoin.c, which partitions the hash table
and avoids cross-partition accesses. I don't think it's directly
applicable :-(, because it chooses the number of partitions in advance,
which is exactly the luxury we do not have here --- if the planner had
gotten the number of entries right then we'd not be needing to spill.
But it might give some ideas.
regards, tom lane