On 21 Jul 2019, at 22:54, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> Chapman Flack <chap@anastigmatix.net> writes:
>> Until now, I had assumed that SFRM_ValuePerCall mode might offer some
>> benefits, such as the possibility of pipelining certain queries and not
>> building up a whole tuplestore in advance.
>
>> But looking in the code, I'm getting the impression that those
>> benefits are only theoretical future ones, as ExecMakeTableFunctionResult
>> implements SFRM_ValuePerCall mode by ... repeatedly calling the function
>> to build up a whole tuplestore in advance.
>
> Yes, that's the case for a SRF in FROM. A SRF in the targetlist
> actually does get the chance to pipeline, if it implements ValuePerCall.
>
> The FROM case could be improved perhaps, if somebody wanted to put
> time into it.
While looking at whether REFCURSOR output could be pipelined into the executor [1], I’ve stumbled upon the same.
By any chance, do either of you know if there are initiatives to make the changes mentioned?
> You'd still need to be prepared to build a tuplestore,
> in case of rescan or backwards fetch; but […]
I’m also interested in your comment here. If the function was STABLE, could not the function scan simply be restarted?
(Ratherthan needing to create the tuplestore for all cases.)
I guess perhaps the backwards scan is where it falls down though...
> […] in principle you could return
> rows immediately while stashing them aside in a tuplestore.
Does the planner have any view on this? When I first saw what was going on, I presumed the planner had decided the cost
ofmultiple function scans was greater than the cost of materialising it in a temporary store.
It occurs to me that, if we made a switch towards pipelining the function scan results directly out, then we might be
looseefficiency where there are a significant number of scans and/or the function cost high. Is that why you were
suggestingto as well stash them aside?
denty.
[1] https://www.postgresql.org/message-id/B2AFCAB5-FACD-44BF-963F-7DD2735FAB5D%40QQdd.eu