Re: [v9.5] Custom Plan API

On 7 May 2014 10:06, Kouhei Kaigai <kaigai@ak.jp.nec.com> wrote:

> Let me list up the things to be clarified / developed randomly.
>
> * Join replacement by FDW; We still don't have consensus about join replacement
>   by FDW. Probably, it will be designed to remote-join implementation primarily,
>   however, things to do is similar. We may need to revisit the Hanada-san's
>   proposition in the past.

Agreed. We need to push down joins into FDWs and we need to push down
aggregates also, so they can be passed to FDWs. I'm planning to look
at aggregate push down.

> * Lookaside for ANY relations; I want planner to try GPU-scan for any relations
>   once installed, to reduce user's administration cost.
>   It needs lookaside allow to specify a particular foreign-server, not foreign-
>   table, then create ForeignScan node that is not associated with a particular
>   foreign-table.

IMHO we would not want to add indexes to every column, on every table,
nor would we wish to use lookaside for all tables. It is a good thing
to be able to add optimizations for individual tables. GPUs are not
good for everything; it is good to be able to leverage their
strengths, yet avoid their weaknesses.

If do you want that, you can write an Event Trigger that automatically
adds a lookaside for any table.

> * ForeignScan node that is not associated with a particular foreign-table.
>   Once we try to apply ForeignScan node instead of Sort or Aggregate, existing
>   FDW implementation needs to be improved. These nodes scan on a materialized
>   relation (generated on the fly), however, existing FDW code assumes
>   ForeignScan node is always associated with a particular foreign-table.
>   We need to eliminate this restriction.

I don't think we need to do that, given the above.

> * FDW method for MultiExec. In case when we can stack multiple ForeignScan
>   nodes, it's helpful to support to exchange scanned tuples in their own
>   data format. Let's assume two ForeignScan nodes are stacked. One performs
>   like Sort, another performs like Scan. If they internally handle column-
>   oriented data format, TupleTableSlot is not a best way for data exchange.

I agree TupleTableSlot may not be best way for bulk data movement. We
probably need to look at buffering/bulk movement between executor
nodes in general, which would be of benefit for the FDW case also.
This would be a problem even for Custom Scans as originally presented
also, so I don't see much change there.

> * Lookaside on the INSERT/UPDATE/DELETE. Probably, it can be implemented
>   using writable FDW feature. Not a big issue, but don't forget it...

Yes, possible.


I hope these ideas make sense. This is early days and there may be
other ideas and much detail yet to come.

-- Simon Riggs                   http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services