Re: Parallel Seq Scan

On Wed, Apr 1, 2015 at 10:28 AM, Amit Kapila <amit.kapila16@gmail.com> wrote:
>> Well, if we *don't* handle it, we're going to need to insert some hack
>> to ensure that the planner doesn't create plans.  And that seems
>> pretty unappealing.  Maybe it'll significantly compromise plan
>> quality, and maybe it won't, but at the least, it's ugly.
>
> I also think changing anything in planner related to this is not a
> good idea, but what about detecting this during execution (into
> ExecFunnel) and then just run the plan locally (by master backend)?

That seems like an even bigger hack; we want to minimize the number of
cases where we create a parallel plan and then don't go parallel.
Doing that in the hopefully-rare case where we manage to blow out the
DSM segments seems OK, but doing it every time a plan of a certain
type gets created doesn't seem very appealing to me.

>> > [1]
>> > I have tried to evaluate what it would take us to support execution
>> > of subplans by parallel workers.  We need to pass the sub plans
>> > stored in Funnel Node (initPlan) and corresponding subplans stored
>> > in planned statement (subplans)  as subplan's stored in Funnel node
>> > has reference to subplans in planned statement.  Next currently
>> > readfuncs.c (functions to read different type of nodes) doesn't support
>> > reading any type of plan node, so we need to add support for reading all
>> > kind
>> > of plan nodes (as subplan can have any type of plan node) and similarly
>> > to execute any type of Plan node, we might need more work
>> > (infrastructure).
>>
>> I don't think you need to do anything that complicated.  I'm not
>> proposing to *run* the initPlan in the workers, just to pass the
>> parameter values down.
>
> Sorry, but I am not able to understand how it will help if just parameters
> (If I understand correctly you want to say about Bitmapset  *extParam;
> Bitmapset  *allParam; in Plan structure) are passed to workers and I
> think they are already getting passed only initPlan and related Subplan
> in planned statement is not passed and the reason is that ss_finalize_plan()
> attaches initPlan to top node (which in this case is Funnel node and not
> PartialSeqScan)
>
> By any chance, do you mean that we run the part of the statement in
> workers and then run initPlan in master backend?

If I'm not confused, it would be the other way around.  We would run
the initPlan in the master backend *first* and then the rest in the
workers.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company