Re: [HACKERS] JIT compiling - v4.0

On 2017-10-04 11:56:47 +0300, Ants Aasma wrote:
> On Wed, Oct 4, 2017 at 9:48 AM, Andres Freund <andres@anarazel.de> wrote:
> > Here's an updated version of the patchset.  There's some substantial
> > changes here, but it's still very obviously very far from committable as
> > a whole. There's some helper commmits that are simple and independent
> > enough to be committable earlier on.
>
> Looks pretty impressive already.

Thanks!


> I wanted to take it for a spin, but got errors about the following
> symbols being missing:
>
> LLVMOrcUnregisterPerf
> LLVMOrcRegisterGDB
> LLVMOrcRegisterPerf
> LLVMOrcGetSymbolAddressIn
> LLVMLinkModules2Needed
>
> As far as I can tell these are not in mainline LLVM. Is there a branch
> or patchset of LLVM available somewhere that I need to use this?

Oops, I'd forgotten about the modifications. Sorry. I've attached them
here.  The GDB and Perf stuff should now be an optional dependency,
too.  The required changes are fairly small, so they hopefully shouldn't
be too hard to upstream.

Please check the git tree for a rebased version of the pg patches, with
a bunch bugfixes (oops, some last minute "cleanups") and performance
fixes.

Here's some numbers for a a TPC-H scale 5 run. Obviously the Q01 numbers
are pretty nice in partcular. But it's also visible that the shorter
query can loose, which is largely due to the JIT overhead - that can be
ameliorated to some degree, but JITing obviously isn't always going to
be a win.

It's pretty impressive that in q01, even after all of this, expression
evaluation *still* is 35% of the total time (25% in the aggregate
transition function). That's partially just because the query does
primarily aggregation, but also because the generated code can stand a
good chunk of improvements.

master q01 min: 14146.498     dev min: 11479.05 [diff -23.24]     dev-jit min: 8659.961 [diff -63.36]
dev-jit-deformmin: 7279.395 [diff -94.34]     dev-jit-deform-inline min: 6997.956 [diff -102.15]
 
master q02 min: 1234.229     dev min: 1208.102 [diff -2.16]     dev-jit min: 1292.983 [diff +4.54]     dev-jit-deform
min:1580.505 [diff +21.91]     dev-jit-deform-inline min: 1809.046 [diff +31.77]
 
master q03 min: 6220.814     dev min: 5424.107 [diff -14.69]     dev-jit min: 5175.125 [diff -20.21]     dev-jit-deform
min:4257.368 [diff -46.12]     dev-jit-deform-inline min: 4218.115 [diff -47.48]
 
master q04 min: 947.476     dev min: 970.608 [diff +2.38]     dev-jit min: 969.944 [diff +2.32]     dev-jit-deform min:
999.006[diff +5.16]     dev-jit-deform-inline min: 1033.78 [diff +8.35]
 
master q05 min: 4729.9     dev min: 4059.665 [diff -16.51]     dev-jit min: 4182.941 [diff -13.08]     dev-jit-deform
min:4147.493 [diff -14.04]     dev-jit-deform-inline min: 4284.473 [diff -10.40]
 
master q06 min: 1603.708     dev min: 1592.107 [diff -0.73]     dev-jit min: 1556.216 [diff -3.05]     dev-jit-deform
min:1516.078 [diff -5.78]     dev-jit-deform-inline min: 1579.839 [diff -1.51]
 
master q07 min: 4549.738     dev min: 4331.565 [diff -5.04]     dev-jit min: 4475.654 [diff -1.66]     dev-jit-deform
min:4645.773 [diff +2.07]     dev-jit-deform-inline min: 4885.781 [diff +6.88]
 
master q08 min: 1394.428     dev min: 1350.363 [diff -3.26]     dev-jit min: 1434.366 [diff +2.78]     dev-jit-deform
min:1716.65 [diff +18.77]     dev-jit-deform-inline min: 1938.152 [diff +28.05]
 
master q09 min: 5958.198     dev min: 5700.329 [diff -4.52]     dev-jit min: 5491.683 [diff -8.49]     dev-jit-deform
min:5582.431 [diff -6.73]     dev-jit-deform-inline min: 5797.475 [diff -2.77]
 
master q10 min: 5228.69     dev min: 4475.154 [diff -16.84]     dev-jit min: 4269.365 [diff -22.47]     dev-jit-deform
min:3767.888 [diff -38.77]     dev-jit-deform-inline min: 3962.084 [diff -31.97]
 
master q11 min: 281.201     dev min: 280.132 [diff -0.38]     dev-jit min: 351.85 [diff +20.08]     dev-jit-deform min:
455.885[diff +38.32]     dev-jit-deform-inline min: 532.093 [diff +47.15]
 
master q12 min: 4289.268     dev min: 4082.359 [diff -5.07]     dev-jit min: 4007.199 [diff -7.04]     dev-jit-deform
min:3752.396 [diff -14.31]     dev-jit-deform-inline min: 3916.653 [diff -9.51]
 
master q13 min: 7110.545     dev min: 6898.576 [diff -3.07]     dev-jit min: 6579.554 [diff -8.07]     dev-jit-deform
min:6304.15 [diff -12.79]     dev-jit-deform-inline min: 6135.952 [diff -15.88]
 
master q14 min: 678.024     dev min: 650.943 [diff -4.16]     dev-jit min: 682.387 [diff +0.64]     dev-jit-deform min:
746.354[diff +9.16]     dev-jit-deform-inline min: 878.437 [diff +22.81]
 
master q15 min: 1641.897     dev min: 1650.57 [diff +0.53]     dev-jit min: 1661.591 [diff +1.19]     dev-jit-deform
min:1821.02 [diff +9.84]     dev-jit-deform-inline min: 1863.304 [diff +11.88]
 
master q16 min: 1890.246     dev min: 1819.423 [diff -3.89]     dev-jit min: 1838.079 [diff -2.84]     dev-jit-deform
min:1962.274 [diff +3.67]     dev-jit-deform-inline min: 2096.154 [diff +9.82]
 
master q17 min: 502.605     dev min: 462.881 [diff -8.58]     dev-jit min: 495.648 [diff -1.40]     dev-jit-deform min:
537.666[diff +6.52]     dev-jit-deform-inline min: 613.144 [diff +18.03]
 
master q18 min: 12863.972     dev min: 11257.57 [diff -14.27]     dev-jit min: 10847.61 [diff -18.59]
dev-jit-deformmin: 10119.769 [diff -27.12]     dev-jit-deform-inline min: 10103.051 [diff -27.33]
 
master q19 min: 281.991     dev min: 264.191 [diff -6.74]     dev-jit min: 331.102 [diff +14.83]     dev-jit-deform
min:373.759 [diff +24.55]     dev-jit-deform-inline min: 531.07 [diff +46.90]
 
master q20 min: 541.154     dev min: 511.372 [diff -5.82]     dev-jit min: 565.378 [diff +4.28]     dev-jit-deform min:
662.926[diff +18.37]     dev-jit-deform-inline min: 805.835 [diff +32.85]
 
master q22 min: 678.266     dev min: 656.643 [diff -3.29]     dev-jit min: 676.886 [diff -0.20]     dev-jit-deform min:
735.058[diff +7.73]     dev-jit-deform-inline min: 943.013 [diff +28.07]
 

master total min: 76772.848     dev min: 69125.71 [diff -11.06]     dev-jit min: 65545.522 [diff -17.13]
dev-jit-deformmin: 62963.844 [diff -21.93]     dev-jit-deform-inline min: 64925.407 [diff -18.25]
 


Greetings,

Andres Freund

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers