Re: postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

On 2016/02/05 17:50, Ashutosh Bapat wrote:

    Btw, IIUC, I think the patch fails to adjust the targetlist of the
    top plan created that way, to output the fdw_scan_tlist, as
    discussed in [1] (ie, I think the attached patch is needed, which is
    created on top of your patch pg_fdw_join_v8.patch).

fdw_scan_tlist represents the output fetched from the foreign server and
is not necessarily the output of ForeignScan. ForeignScan node's output
is represented by tlist argument to.

1119     return make_foreignscan(tlist,
1120                             local_exprs,
1121                             scan_relid,
1122                             params_list,
1123                             fdw_private,
1124                             fdw_scan_tlist,
1125                             remote_exprs,
1126                             outer_plan);

This tlist is built using build_path_tlist() for all join plans. IIUC,
all of them output the same targetlist. We don't need to make sure that
targetlist match as long as we are using the targetlist passed in by
create_scan_plan(). Do you have a counter example?

Maybe my explanation was not correct, but I'm saying that the targertlist of the above outer_plan should be set to the fdw_scan_tlist, to avoid misbehavior.  Here is such an example (add() in the example is a user defined function that simply adds two arguments, defined by: create function add(integer, integer) returns integer as '/path/to/func', 'add' language c strict):

postgres=# create foreign table foo (a int) server myserver options (table_name 'foo');
postgres=# create foreign table bar (a int) server myserver options (table_name 'bar');
postgres=# create table tab (a int, b int);
postgres=# insert into foo select a from generate_series(1, 1000) a;
postgres=# insert into bar select a from generate_series(1, 1000) a;
postgres=# insert into tab values (1, 1);
postgres=# analyze foo;
postgres=# analyze bar;
postgres=# analyze tab;

[Terminal 1]
postgres=# begin;
BEGIN
postgres=# update tab set b = b + 1 where a = 1;
UPDATE 1

[Terminal 2]
postgres=# explain verbose select tab.* from tab, foo, bar where foo.a = bar.a and add(foo.a, bar.a) > 0 limit 1 for update;

                QUERY PLAN

--------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------
 Limit  (cost=100.00..107.70 rows=1 width=70)
   Output: tab.a, tab.b, tab.ctid, foo.*, bar.*
   ->  LockRows  (cost=100.00..2663.48 rows=333 width=70)
         Output: tab.a, tab.b, tab.ctid, foo.*, bar.*
         ->  Nested Loop  (cost=100.00..2660.15 rows=333 width=70)
               Output: tab.a, tab.b, tab.ctid, foo.*, bar.*
               ->  Foreign Scan  (cost=100.00..2654.97 rows=333 width=56)
                     Output: foo.*, bar.*
                     Filter: (add(foo.a, bar.a) > 0)
                     Relations: (public.foo) INNER JOIN (public.bar)
                     Remote SQL: SELECT ROW(r2.a), ROW(r3.a), r2.a, r3.a FROM (public.foo r2 INNER JOIN public.bar r3 ON (TRUE)) WHERE ((r2.a = r3.a)) F
OR UPDATE OF r2 FOR UPDATE OF r3
                     ->  Hash Join  (cost=247.50..301.25 rows=333 width=56)
                           Output: foo.*, bar.*
                           Hash Cond: (foo.a = bar.a)
                           Join Filter: (add(foo.a, bar.a) > 0)
                           ->  Foreign Scan on public.foo (cost=100.00..135.00 rows=1000 width=32)
                                 Output: foo.*, foo.a
                                 Remote SQL: SELECT a FROM public.foo FOR UPDATE
                           ->  Hash  (cost=135.00..135.00 rows=1000 width=32)
                                 Output: bar.*, bar.a
                                 ->  Foreign Scan on public.bar (cost=100.00..135.00 rows=1000 width=32)
                                       Output: bar.*, bar.a
                                       Remote SQL: SELECT a FROM public.bar FOR UPDATE
               ->  Materialize  (cost=0.00..1.01 rows=1 width=14)
                     Output: tab.a, tab.b, tab.ctid
                     ->  Seq Scan on public.tab  (cost=0.00..1.01 rows=1 width=14)
                           Output: tab.a, tab.b, tab.ctid
(27 rows)

postgres=# select tab.* from tab, foo, bar where foo.a = bar.a and add(foo.a, bar.a) > 0 limit 1 for update;

[Terminal 1]
postgres=# commit;
COMMIT

[Terminal 2] (After the commit in Terminal 1, Terminal 2 will show the following.)
 a | b
---+---
(0 rows)

This is wrong.  (Note that since the SELECT FOR UPDATE doesn't impose any condition on a tuple from the local table tab, the EvalPlanQual recheck executed should succeed.)  The reason for that is that the targetlist of the local join plan is the same as for the ForeignScan, which outputs neither foo.a nor bar.a required as an argument of the function add().