Re: [HACKERS] asynchronous execution

I noticed that this patch is conflicting with 665d1fa (Logical
replication) so I rebased this. Only executor/Makefile
conflicted.

At Mon, 31 Oct 2016 10:39:12 +0900 (JST), Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp> wrote in
<20161031.103912.217430542.horiguchi.kyotaro@lab.ntt.co.jp>
> This a PoC patch of asynchronous execution feature, based on a
> executor infrastructure Robert proposed. These patches are
> rebased on the current master.
> 
> 0001-robert-s-2nd-framework.patch
> 
>  Roberts executor async infrastructure. Async-driver nodes
>  register its async-capable children and sync and data transfer
>  are done out of band of ordinary ExecProcNode channel. So async
>  execution no longer disturbs async-unaware node and slows them
>  down.
> 
> 0002-Fix-some-bugs.patch
> 
>  Some fixes for 0001 to work. This is just to preserve the shape
>  of 0001 patch.
> 
> 0003-Modify-async-execution-infrastructure.patch
> 
>  The original infrastructure doesn't work when multiple foreign
>  tables is on the same connection. This makes it work.
> 
> 0004-Make-postgres_fdw-async-capable.patch
> 
>  Makes postgres_fdw to work asynchronously.
> 
> 0005-Use-resource-owner-to-prevent-wait-event-set-from-le.patch
> 
>  This addresses a problem pointed by Robers about 0001 patch,
>  that WaitEventSet used for async execution can leak by errors.
> 
> 0006-Apply-unlikely-to-suggest-synchronous-route-of-ExecA.patch
> 
>  ExecAppend gets a bit slower by penalties of misprediction of
>  branches. This fixes it by using unlikely() macro.
> 
> 0007-Add-instrumentation-to-async-execution.patch
> 
>  As the description above for 0001, async infrastructure conveys
>  tuples outside ExecProcNode channel so EXPLAIN ANALYZE requires
>  special treat to show sane results. This patch tries that.
> 
> 
> A result of a performance measurement is in this message.
> 
> https://www.postgresql.org/message-id/20161025.182150.230901487.horiguchi.kyotaro@lab.ntt.co.jp
> 
> 
> | t0  - SELECT sum(a) FROM <local single table>;
> | pl  - SELECT sum(a) FROM <4 local children>;
> | pf0 - SELECT sum(a) FROM <4 foreign children on single connection>;
> | pf1 - SELECT sum(a) FROM <4 foreign children on dedicate connections>;
> ...
> | async
> |   t0: 3885.84 ( 40.20)  0.86% faster (should be error but stable on my env..)
> |   pl: 1617.20 (  3.51)  1.26% faster (ditto)
> |  pf0: 6680.95 (478.72)  19.5% faster
> |  pf1: 1886.87 ( 36.25)  77.1% faster


regards,

-- 
Kyotaro Horiguchi
NTT Open Source Software Center