On Tue, Sep 27, 2016 at 3:08 PM, Robert Haas <robertmhaas@gmail.com> wrote:
> I don't know what any of that means. You said we need something like
> an LWLock, but I think we don't. The workers just write the results
> of their own final merges into shm_mqs. The leader can read from any
> given shm_mq until no more tuples can be read without blocking, just
> like nodeGather.c does, or at least it can do that unless its own
> queue fills up first. No mutual exclusion mechanism is required for
> any of that, as far as I can see - not an LWLock, and not anything
> similar.
I'm confused about the distinction you're making. Isn't the shm_mqs
queue itself a mutual exclusion mechanism? The leader cannot really do
anything without some input to process, because of the dependency that
naturally exists. ISTM that everything else we've discussed is
semantics, and/or about particular mechanisms in Postgres.
At a high level, based on my intuition about performance
characteristics, I have reservations about eager merging in the
leader. That's all I mean. There is a trade-off to be made between
eager and lazy merging. As I pointed out, the author of the Volcano
paper (Goetz Graefe) went on at length about this particular trade-off
for parallel sort almost 30 years ago. So, it's not clear that that
would be an easy win, or even a win.
That's all I meant.
--
Peter Geoghegan