Re: Custom Scan APIs (Re: Custom Plan node)

> On Tue, Apr 15, 2014 at 10:44 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > Robert Haas <robertmhaas@gmail.com> writes:
> >> On Mon, Apr 14, 2014 at 4:43 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >>> Yeah.  After a fast review of the custom-scan and cache-scan
> >>> patches, it seems to me that my original fears are largely
> >>> confirmed: the custom scan patch is not going to be sufficient to
> >>> allow development of any truly new plan type.  Yeah, you can plug in
> >>> some new execution node types, but actually doing anything
> >>> interesting is going to require patching other parts of the system.
> >
> >> Without prejudice to the rest of what you said, this argument doesn't
> >> hold much water with me.  I mean, anything that our extensibility
> >> mechanism doesn't support today will require new hooks, but does that
> >> mean we're never going to add any more hooks?  I sure hope not.
> >
> > No, that's not what I said.  ISTM that the argument for the
> > custom-scan API is that it allows interesting new things to be done
> > *without further modifying the core code*.  But the example
> > application (cache_scan) fails to demonstrate that, and indeed seems
> > to be a counterexample.  Whether we'd accept cache_scan on its own
> > merits is a separate question.  The problem for me is that custom-scan
> > isn't showing that it can support what was claimed without doing
> > serious damage to modularity and maintainability of the core code.
> 
> I think there's two separate things in there, one of which I agree with
> and one of which I disagree with.  I agree that we must avoid damaging the
> modularity and maintainability of the core code; I don't agree that
> custom-scan needs to be able to do interesting things with zero additional
> changes to the core code.  If we come up with three interesting applications
> for custom scan that require 5 new hooks between them, I'll consider that
> a major success - assuming those hooks don't unduly limit future changes
> we may wish to make in the core code.  I think your concern about exposing
> APIs that may not be terribly stable is well-founded, but I don't think
> that means we shouldn't expose *anything*.
> 
I agree 100%.

We usually change hook definition release-by-release, and it is author's
responsibility to follow the newer interface if he continues to maintain
his extension on the newer release also.
Probably, it is a gray stuff neither black nor white. If we can design
a perfect interface, it might be good but has no evolution further.
Of course, it does not justify poor designed interface, but an important
stuff is to find out a best way at this moment. It may take core
refactoring, not just exposing static functions. What I tried to implement
is the only way to implement it.

Thanks,
--
NEC OSS Promotion Center / PG-Strom Project
KaiGai Kohei <kaigai@ak.jp.nec.com>