Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [SenderAddress Forgery]Re: [HACKERS] path toward faster partition pruning

On 18 January 2018 at 23:56, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp> wrote:
>> I've not fully worked out how run-time pruning
>> will use this as it'll need another version of
>> get_partitions_from_clauses but passes in a PartScanClauseInfo
>> instead, and does not call extract_partition_key_clauses. That area
>> probably  needs some shuffling around so that does not end up a big
>> copy and paste of all that new logic.
>>
> So, I've been assuming that the planner changes in the run-time pruning
> patch have to do with selecting clauses (restriction clauses not
> containing Consts and/or join clauses) to be passed to the executor by
> recording them in the Append node.  Will they be selected by the planner
> calling into partition.c?

I had thought so. I only have a rough idea in my head and that's that
the PartitionPruneInfo struct that I wrote for the run-time pruning
patch would have the clause List replaced with this new
PartScanClauseInfo struct (which likely means it needs to go into
primnodes.h), this struct would contain all the partition pruning
clauses in a more structured form so that no matching of quals to the
partition key wouldn't be required during execution. The idea is that
we'd just need to call; remove_redundant_clauses,
extract_bounding_datums and get_partitions_for_keys. I think this is
the bare minimum of work that can be done in execution since we can't
remove the redundant clauses until we know the values of the Params.

Likely this means there will need to be 2 functions externally
accessible for this in partition.c. I'm not sure exactly how best to
do this. Maybe it's fine just to have allpaths.c call
extract_partition_key_clauses to generate the PartScanClauseInfo then
call some version of get_partitions_from_clauses which does do the
extract_partition_key_clauses duties. This is made more complex by the
code that handles adding the default partition bound to the quals. I'm
not yet sure where that should live.

I've also been thinking of having some sort of PartitionPruneContext
struct that we can pass around the functions. Runtime pruning had to
add structs which store the Param values to various functions which I
didn't like. It would be good to just add those to the context and
have them passed down without having to bloat the parameters in the
functions. I might try and do that tomorrow too. This should make the
footprint of the runtime pruning patch is a bit smaller.

> Meanwhile, here is a revised version (v21) that incorporates your changes.
>  I added you as the author in 0002 and 0005 patches.  I guess a v22 will
> have to follow very soon...

Thanks for merging that in.

I'll have a try at making this work tomorrow, although I'm not sure
yet how much time I'll have to dedicate to it as I have a few other
things to do too.

-- 
 David Rowley                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services