Re: Foreign join pushdown vs EvalPlanQual

On 2015/07/07 19:15, Etsuro Fujita wrote:
> On 2015/07/06 9:42, Kouhei Kaigai wrote:
>> However, we need to pay attention on advantages towards the alternatives.
>> Hooks on add_paths_to_joinrel() enables to implement identical stuff,
>> with
>> less complicated logic to reproduce left / right relations from
>> RelOptInfo
>> of the joinrel. (Note that RelOptInfo->fdw_private enables to avoid path-
>> construction multiple times.)
>> I'm uncertain why this API change is necessary to fix up the problem
>> around EvalPlanQual.
>
> Yeah, maybe we wouldn't need any API change.  I think we would be able
> to fix this by complicating add_path as you pointed out upthread.  I'm
> not sure that complicating it is a good idea, though.  I think that it
> might be possible that the callback in standard_join_search would allow
> us to fix this without complicating the core path-cost-comparison stuff
> such as add_path.  I noticed that what I proposed upthread doesn't work
> properly, though.

To resolve this issue, I tried to make the core create an alternative 
plan that will be used in an EvalPlanQual recheck, instead of a foreign 
scan that performs a foreign join remotely (ie, scanrelid = 0).  But I 
changed that idea.  Instead, I'd like to propose that it's the FDW's 
responsibility to provide such a plan.  Specifically, I'd propose that 
(1) we add a new Path field, say subpath, to the ForeignPath data 
structure and that (2) when generating a ForeignPath node for a foreign 
join, an FDW must provide the subpath Path node by itself.  As before, 
it'd be recommended to use

ForeignPath *
create_foreignscan_path(PlannerInfo *root, RelOptInfo *rel,                        double rows, Cost startup_cost, Cost
total_cost,                       List *pathkeys,                        Relids required_outer,
Path*subpath,                        List *fdw_private)
 

where subpath is the subpath Path node that has the pathkeys and the 
required_outer rels.  (subpath is NULL if scanning a base relation.) 
Also, it'd be recommended that an FDW generates such ForeignPath nodes 
by considering, for each of paths in the rel's pathlist, whether to push 
down that path (to generate a ForeignPath node for a foreign join that 
has the same pathkeys and parameterization as that path).  So, if 
generating the ForeignPath node, that path could be used as the subpath 
Path node.

(I think the current postgres_fdw patch only considers an unsorted, 
unparameterized path for performing a foreign join remotely, but I think 
we should also consider presorted and/or parameterized paths.)

I think this idea would apply to the API location that you proposed. 
However, ISTM that this idea would work better for the API call from 
standard_join_search because the rel's pathlist at that point has more 
paths worthy of consideration, in view of not only costs and sizes but 
pathkeys and parameterization.

I think the subplan created from the subpath Path node could be used in 
an EvalPlanQual recheck, instead of a foreign scan that performs a 
foreign join remotely, as discussed previously.

Comments welcome!

Best regards,
Etsuro Fujita