pgsql: Code review for foreign/custom join pushdown patch.
pgsql: Code review for foreign/custom join pushdown patch.
От:
Tom Lane <tgl@sss.pgh.pa.us>
Дата:
Code review for foreign/custom join pushdown patch. Commit e7cb7ee14555cc9c5773e2c102efd6371f6f2005 included some design decisions that seem pretty questionable to me, and there was quite a lot of stuff not to like about the documentation and comments. Clean up as follows: * Consider foreign joins only between foreign tables on the same server, rather than between any two foreign tables with the same underlying FDW handler function. In most if not all cases, the FDW would simply have had to apply the same-server restriction itself (far more expensively, both for lack of caching and because it would be repeated for each combination of input sub-joins), or else risk nasty bugs. Anyone who's really intent on doing something outside this restriction can always use the set_join_pathlist_hook. * Rename fdw_ps_tlist/custom_ps_tlist to fdw_scan_tlist/custom_scan_tlist to better reflect what they're for, and allow these custom scan tlists to be used even for base relations. * Change make_foreignscan() API to include passing the fdw_scan_tlist value, since the FDW is required to set that. Backwards compatibility doesn't seem like an adequate reason to expect FDWs to set it in some ad-hoc extra step, and anyway existing FDWs can just pass NIL. * Change the API of path-generating subroutines of add_paths_to_joinrel, and in particular that of GetForeignJoinPaths and set_join_pathlist_hook, so that various less-used parameters are passed in a struct rather than as separate parameter-list entries. The objective here is to reduce the probability that future additions to those parameter lists will result in source-level API breaks for users of these hooks. It's possible that this is even a small win for the core code, since most CPU architectures can't pass more than half a dozen parameters efficiently anyway. I kept root, joinrel, outerrel, innerrel, and jointype as separate parameters to reduce code churn in joinpath.c --- in particular, putting jointype into the struct would have been problematic because of the subroutines' habit of changing their local copies of that variable. * Avoid ad-hocery in ExecAssignScanProjectionInfo. It was probably all right for it to know about IndexOnlyScan, but if the list is to grow we should refactor the knowledge out to the callers. * Restore nodeForeignscan.c's previous use of the relcache to avoid extra GetFdwRoutine lookups for base-relation scans. * Lots of cleanup of documentation and missed comments. Re-order some code additions into more logical places. Branch ------ master Details ------- http://git.postgresql.org/pg/commitdiff/1a8a4e5cde2b7755e11bde2ea7897bd650622d3e Modified Files -------------- contrib/file_fdw/file_fdw.c | 3 +- contrib/postgres_fdw/postgres_fdw.c | 3 +- doc/src/sgml/custom-scan.sgml | 134 ++++++------ doc/src/sgml/fdwhandler.sgml | 152 +++++++++---- src/backend/commands/explain.c | 4 +- src/backend/executor/execScan.c | 23 +- src/backend/executor/nodeCustom.c | 55 +++-- src/backend/executor/nodeForeignscan.c | 41 ++-- src/backend/executor/nodeIndexonlyscan.c | 6 +- src/backend/foreign/foreign.c | 42 ++-- src/backend/nodes/copyfuncs.c | 8 +- src/backend/nodes/outfuncs.c | 9 +- src/backend/optimizer/path/joinpath.c | 342 +++++++++++------------------- src/backend/optimizer/plan/createplan.c | 83 +++----- src/backend/optimizer/plan/setrefs.c | 256 +++++++++++----------- src/backend/optimizer/plan/subselect.c | 2 + src/backend/optimizer/util/plancat.c | 7 +- src/backend/optimizer/util/relnode.c | 27 ++- src/backend/utils/adt/ruleutils.c | 19 +- src/include/executor/executor.h | 1 + src/include/foreign/fdwapi.h | 27 ++- src/include/nodes/plannodes.h | 37 +++- src/include/nodes/primnodes.h | 10 +- src/include/nodes/relation.h | 45 +++- src/include/optimizer/paths.h | 14 +- src/include/optimizer/planmain.h | 4 +- 26 files changed, 701 insertions(+), 653 deletions(-)
Re: pgsql: Code review for foreign/custom join pushdown patch.
От:
Andres Freund <andres@anarazel.de>
Дата:
On 2015-05-10 18:36:45 +0000, Tom Lane wrote: > * Consider foreign joins only between foreign tables on the same server, > rather than between any two foreign tables with the same underlying FDW > handler function. In most if not all cases, the FDW would simply have had > to apply the same-server restriction itself (far more expensively, both for > lack of caching and because it would be repeated for each combination of > input sub-joins), or else risk nasty bugs. Anyone who's really intent on > doing something outside this restriction can always use the > set_join_pathlist_hook. Hm. I'm not sure I can agree with that judgement. It does not seem that far fetched to want to push joins between two different FDW servers to one FDW. Especially with inheritance now being allowed it's quite possible people will come up with complicated partitioning scheme where it's good where the server the user is interacting with does as little as possible. Possibly it'll also allow to reduce slow network traffic. Greetings, Andres Freund
Re: pgsql: Code review for foreign/custom join pushdown patch.
От:
Tom Lane <tgl@sss.pgh.pa.us>
Дата:
Andres Freund writes: > On 2015-05-10 18:36:45 +0000, Tom Lane wrote: >> * Consider foreign joins only between foreign tables on the same server, >> rather than between any two foreign tables with the same underlying FDW >> handler function. In most if not all cases, the FDW would simply have had >> to apply the same-server restriction itself (far more expensively, both for >> lack of caching and because it would be repeated for each combination of >> input sub-joins), or else risk nasty bugs. Anyone who's really intent on >> doing something outside this restriction can always use the >> set_join_pathlist_hook. > Hm. I'm not sure I can agree with that judgement. It does not seem that > far fetched to want to push joins between two different FDW servers to > one FDW. What I doubt is that people can do that *correctly*. See my counterexamples in the pgsql-hackers thread. In any case, as also noted on that thread, an FDW that wants to adjust this behavior could do so by changing RelOptInfo.serverid to match the ID of the selected common server during GetForeignPaths. regards, tom lane