Kaigai-san,
On 2019/05/29 12:13, Kohei KaiGai wrote:
> One interesting data type in Apache Arrow is "Struct" data type. It is
> equivalent to composite
> type in PostgreSQL. The "Struct" type has sub-fields, and individual
> sub-fields have its own
> values array for each.
>
> It means we can skip to load the sub-fields unreferenced, if
> query-planner can handle
> referenced and unreferenced sub-fields correctly.
> On the other hands, it looks to me RelOptInfo or other optimizer
> related structure don't have
> this kind of information. RelOptInfo->attr_needed tells extension
> which attributes are referenced
> by other relation, however, its granularity is not sufficient for sub-fields.
Isn't that true for some other cases as well, like when a query accesses
only some sub-fields of a json(b) column? In that case too, planner
itself can't optimize away access to other sub-fields. What it can do
though is match a suitable index to the operator used to access the
individual sub-fields, so that the index (if one is matched and chosen)
can optimize away accessing unnecessary sub-fields. IOW, it seems to me
that the optimizer leaves it up to the indexes (and plan nodes) to further
optimize access to within a field. How is this case any different?
> Probably, all we can do right now is walk-on the RelOptInfo list to
> lookup FieldSelect node
> to see the referenced sub-fields. Do we have a good idea instead of
> this expensive way?
> # Right now, PG-Strom loads all the sub-fields of Struct column from
> arrow_fdw foreign-table
> # regardless of referenced / unreferenced sub-fields. Just a second best.
I'm missing something, but if PG-Strom/arrow_fdw does look at the
FieldSelect nodes to see which sub-fields are referenced, why doesn't it
generate a plan that will only access those sub-fields or why can't it?
Thanks,
Amit