Re: ctidscan as an example of custom-scan (Re: [v9.5] Custom Plan API)

> -----Original Message-----
> From: Tom Lane [mailto:tgl@sss.pgh.pa.us]
> Sent: Wednesday, July 15, 2015 5:47 AM
> To: Robert Haas
> Cc: Alvaro Herrera; hlinnaka; Kaigai Kouhei(海外 浩平); Michael Paquier; Jim
> Nasby; Kohei KaiGai; PgHacker; Simon Riggs
> Subject: Re: ctidscan as an example of custom-scan (Re: [HACKERS] [v9.5] Custom
> Plan API)
>
> Robert Haas <robertmhaas@gmail.com> writes:
> > On Tue, Jul 14, 2015 at 3:07 PM, Alvaro Herrera
> > <alvherre@2ndquadrant.com> wrote:
> >> As a general principle, I think it's a good idea to have a module that's
> >> mostly just a skeleton that guides people into writing something real to
> >> use whatever API is being tested.  It needs to be simple enough that not
> >> much need to be deleted when writing the real thing, and complex enough
> >> to cover the parts that need covering.  If whatever replaces ctidscan is
> >> too complex, it will not serve that purpose.
> >>
> >> My guess is that something whose only purpose is to test the custom scan
> >> interface for coverage purposes can be simpler than this module.
>
> > See, I actually think the opposite: I think we've been accumulating a
> > reasonable amount of test code that actually serves no really useful
> > purpose and is just cruft.  Stuff like test_shm_mq and test_decoding
> > seem like they actually catches bugs, so I like that, but I think
> > stuff like worker_spi is actually TOO simple to be useful in building
> > anything real, and it provides no useful test coverage, either.  But
> > this is all a matter of opinion, of course, and I'll defer to whatever
> > the consensus is.
>
> I think this ties into my core unhappiness with the customscan stuff,
> which is that I don't believe it's *possible* to do anything of very
> great interest with it.  I think anything really useful will require
> core code modifications and/or hooks that don't exist now.  So a finger
> exercise like ctidscan, even though it might have some marginal use,
> doesn't do much to alleviate that concern.  It certainly doesn't seem
> like it's a suitable placeholder proving that we aren't breaking any
> actual use cases for the feature.
>
The ctidscan is originally designed to validate the behavior of custom-scan
interface, so it is natural this module is valuable in limited cased.

However, I don't think that anything valuable usually takes core code
enhancement and/or new hooks, because we already have various extensions
around core code that utilizes existing infrastructures (even though its
specifications may be changed on major version up).
At least, custom-scan enables to implement edge-features which are not
easy to merge the core because of various reasons; like dependency to
proprietary library, too experimental features, too large code to review
as minimum valuable portion and so on.

> (BTW, if we care about the use cases this has, such as data recovery from
> partially-corrupt tables, it would make way more sense and take way less
> net new code to just teach TidScan about it.)
>
What I discussed with Hanada-san before was, a custom-scan provider that
replaces a particular relations join by simple scan of materialized-view
transparently. It is probably one other use case. Its design is in my
brain, but time for development is missing piece for me.

Thanks,
--
NEC Business Creation Division / PG-Strom Project
KaiGai Kohei <kaigai@ak.jp.nec.com>