Обсуждение: ecxt_scantuple has wrong TupleDesc

Hi all,

I wonder why my ecxt_scantuple has a TupleDesc matching the subplan's tlist, not my plan's tlist. Basically, my
questionis what is x in
 

econtext->ecxt_scantuple = x ?

I have written a new executor node Foo, with corresponding ExecFoo and make_foo functions. I have also written a new
Exprtype called Bar, along with a ExecEvalBar. 
 

In make_foo I append some Bar columns. When I try to evaluate the Bar columns in ExecFoo via ExecProject, I need the
Barcolumns to access each other. 
 

But sadly, the ecxt_scantuple doesn't have any Bar columns because it has a TupleDesc from the subplan.

What are the steps I have to take to make sure I can access my Bar columns (which only exist in Foo) in ExecEvalBar?
Wheredo I get a TupleTableSlot with the desired TupleDesc from?
 

I'll be happy to provide more information, if needed.

Regards
Peter

peter.trautmeier@gmx.de writes:
> I wonder why my ecxt_scantuple has a TupleDesc matching the subplan's
> tlist, not my plan's tlist.

That's the way it's supposed to be --- the scantuple slot is for
scanning your subplan's output.

> I have written a new executor node Foo, with corresponding ExecFoo and make_foo functions. I have also written a new
Exprtype called Bar, along with a ExecEvalBar. 
 

> In make_foo I append some Bar columns. When I try to evaluate the Bar columns in ExecFoo via ExecProject, I need the
Barcolumns to access each other. 
 

That makes no sense at all.  ExecProject can't be expected to access
output columns of the current node --- they haven't been computed yet.
        regards, tom lane

Thanks Tom, 

that made it clear I made a mistake.

> That's the way it's supposed to be --- the scantuple slot is for
> scanning your subplan's output.

Browsing through the code I get the impression, that

- ecxt_scantuple is only used by Scan nodes (i.e. SeqScan, IndexScan, SubqueryScan and FunctionScan)

- ecxt_innertuple and ecxt_outertuple are used by Join nodes (but they don't use the ecxt_scantuple at all).

Is it right that the ecxt_scantuple is only used by leafnodes (Scans) and never by Joins?

> > I have written a new executor node Foo, with corresponding ExecFoo and
> make_foo functions. I have also written a new Expr type called Bar, along
> with a ExecEvalBar. 
> 
> > In make_foo I append some Bar columns. When I try to evaluate the Bar
> columns in ExecFoo via ExecProject, I need the Bar columns to access each
> other. 
> 
> That makes no sense at all.  ExecProject can't be expected to access
> output columns of the current node --- they haven't been computed yet.

Conceptionally my only way is then to crack the dependency between the Bar columns by splitting the computation in
differentnodes - and then put these nodes atop of each other.
 

But then, I still don't get the relationship between
INNER, OUTER varnos on the one side and ecxt_scantuple, ecxt_outertuple and ecxt_innertuple on the other side.

May a non-leaf node refer to a Var with a 'normal' scan varno or only to INNER and OUTER varnos?

How is my Foo node supposed to access the suplan's (whatever that might me, a Scan or a Join) columns? Should Foo'-
tlisthave Vars with OUTER varnos and the right varattos?
 

I cannot find an example of a node that (1) does projection and (2) is not a Scan (i.e a leaf node) and (3) uses
ecxt_scantupleto pipe information to an upper node.
 

I'd be happy if you clarified my misconceptions, I greatly appreciate your help.

Regards
Peter

On Tue, Dec 18, 2007 at 02:48:23PM +0100, peter.trautmeier@gmx.de wrote:
> But then, I still don't get the relationship between
>
>  INNER, OUTER varnos on the one side and
>  ecxt_scantuple, ecxt_outertuple and ecxt_innertuple on the other side.
>
> May a non-leaf node refer to a Var with a 'normal' scan varno or only to INNER and OUTER varnos?

They're just variables. When you call a subnode it returns a
tupletableslot. When you want to return a tuple to your parent, you
just return it.

> How is my Foo node supposed to access the suplan's (whatever that
> might me, a Scan or a Join) columns? Should Foo'- tlist have Vars
> with OUTER varnos and the right varattos?

You call your subplan: mysubplantuple = ExecProcNode(mysubplan). Vars
and VarNos are just if you want to use ExecProject (which is not
required).

> I cannot find an example of a node that (1) does projection and (2)
> is not a Scan (i.e a leaf node) and (3) uses ecxt_scantuple to pipe
> information to an upper node.

You don't use ecxt_scantuple to pipe to your parent, you create a new
tuple based on the old ones, and simply "return" it. You don't have to
use the ecxt_* fields at all, for example, the ExecAppend node. It's
just convenient to reuse fields that are used for a common purpose.

Have a nice day,
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> Those who make peaceful revolution impossible will make violent revolution inevitable.
>  -- John F Kennedy

Martijn van Oosterhout <kleptog@svana.org> writes:
> On Tue, Dec 18, 2007 at 02:48:23PM +0100, peter.trautmeier@gmx.de wrote:
>> But then, I still don't get the relationship between
>> 
>> INNER, OUTER varnos on the one side and 
>> ecxt_scantuple, ecxt_outertuple and ecxt_innertuple on the other side.
>> 
>> May a non-leaf node refer to a Var with a 'normal' scan varno or only to INNER and OUTER varnos?

> They're just variables. When you call a subnode it returns a
> tupletableslot. When you want to return a tuple to your parent, you
> just return it.

More to the point, this is all dependent on how the planner's setrefs.c
sets up the varnos in the finished plan tree; those varnos will
determine which slot ExecEvalVar looks in at runtime.  Historically
there was some inconsistency in how single-input non-leaf plan nodes
did things --- looking at 8.2, Agg and Group nodes got their Vars set up
as varno 0 (compelling the use of scantuple to hold the subplan output
tuple at runtime), Result got set up with varno OUTER (compelling use of
outertuple), and it looks like none of the other plan node types in this
category do expression evaluation at all.  As of 8.3 I made all three use
varno OUTER in order to simplify setrefs.c a tad.  It would have been
about equally consistent to use varno 0 across the board; I think I did
it this way because it made fewer cases for the deparsing code in
ruleutils.c to deal with.

None of this seems to have much to do with Peter's real problem though;
he's looking for a way to get at values that are being computed in the
current plan node, not by its input.  I think he needs to invent a
back-channel for his special functions and node type to communicate
through.  Perhaps something similar to the way aggregate functions can
communicate with nodeAgg.c via AggState in fcinfo->context would work.
        regards, tom lane