Re: Parallel bitmap heap scan

On Sun, Nov 27, 2016 at 3:34 PM, Dilip Kumar <dilipbalaut@gmail.com> wrote:
> On Sun, Nov 27, 2016 at 3:15 AM, Robert Haas <robertmhaas@gmail.com> wrote:
>> I think the Barrier stuff has a process for choosing one worker to
>> conduct a particular phase.  So it seems like if the Barrier API is
>> well-designed, you should be able to use it to decide who will conduct
>> the index scan, and then when that's done everyone can proceed to
>> scanning the heap.  If that can't work for some reason, Thomas should
>> probably adjust his API so it does.  He's presenting that as a
>> generally-useful primitive...
>
> If I understand the barrier API correctly, It has two Part.
> 1. BarrierInit  2. BarrierWait.
>
> 1. In BarrierInit we defined that, how many worker(lets say nworkers)
> should cross the barrier, before we are allowed to cross the
> BarriedWait.
>
> 2. BarrierWait, will actually make calling process wait until
> BarrierWait is not called for nworkers times.
>
> So I am not very clear, If we call BarrierInit with nworkers=1, then
> first question is when should we call BarrierWait, because as soon as
> we call BarrierWait count will reach 1, and now everyone is allowed to
> proceed. so obviously It should be called once the Bitmap is Ready.
>
> Second question is, if it's called only after Bitmap is ready, then
> what about other process, how they are supposed to wait until bitmap
> is not ready. If they wait using BarrierWait, it again make the count
> 1 and everyone is allowed to proceed. Which doesn't seems correct.
>
> Correct me if I am missing something ?

I'm not sure if it's the right tool for this job or not and haven't
studied this patch yet.  I will.  But here is one way to use barrier.c
for something like this, based on the description above.  It's
slightly more complicated than you said because you don't know whether
the leader is going to participate or how many of the planned workers
will actually be able to start up, so there would be no way to provide
that 'participants' argument to BarrierInit and any given participant
might already have missed some of the 'BarrierWait' calls by the time
it starts running, so merely calling BarrierWait the right number of
times isn't enough to stay in sync.  So instead you do this:
   #define PBS_PHASE INIT 0   #define PBS_PHASE_BUILDING 1   #define PBS_PHASE_SCANNING 2

Initialise the barrier with BarrierInit(&something->barrier, 0), which
says that you don't know how many participants there will be.

Somewhere in each participant you need to do this exactly once:
   BarrierAttach(&something->barrier);

I think you need to track whether you've called that yet and do so on
demand in your ExecBitmapHeap function.  You can't just do it in
ExecBitmapHeapInitializeWorker because the leader needs to do it too,
but *only* if it runs the plan.  Then you need something like this:
   switch (BarrierPhase(&something->barrier)   {   case PBS_PHASE_INIT:       if (BarrierWait(&something->barrier,
WAIT_EVENT_PBS_PHASE_INIT))      {           /* Serial phase that will run in only one chosen participant. */
build_the_bitmap();      }       /* Fall through. */
 
   case PBS_PHASE_BUILDING:       BarrierWait(&something->barrier, WAIT_EVENT_PBS_PHASE_BUILDING);       /* Fall
through.*/
 
   case PBS_PHASE_SCANNING:       scan_the_bitmap_and_emit_one_tuple();   }

When a new participant arrives here, if it finds that we're still in
the INIT phase, then it enters an election to see if it can build the
bitmap; one lucky participant wins and does that, while any other
participants twiddle their thumbs at the next BarrierWait call.  If  a
new participant finds that we're already in the BUILDING phase when it
arrives, then it has missed that election and just has to wait for the
building to be completed.  Once they all agree that building has
finished, we move onto scanning.  If a new arrival finds that we're in
SCANNING phase, then it happily scans and emits tuples.  Does that
make sense?

Not sure exactly how to coordinate rescans yet, but probably with
BarrierWaitSet(&something->barrier, PBS_PHASE_INIT).

-- 
Thomas Munro
http://www.enterprisedb.com