Re: JIT compiling with LLVM v11

On 3/1/18 03:02, Andres Freund wrote:
> I've pushed a revised version of my JIT patchset.
> The git tree is at
>   https://git.postgresql.org/git/users/andresfreund/postgres.git
> in the jit branch
>   https://git.postgresql.org/gitweb/?p=users/andresfreund/postgres.git;a=shortlog;h=refs/heads/jit

(testing 2e15e8b8100a61ec092a1e5b2db4a93f07a64cbd)

I'm having an interesting time getting this to build on macOS.

First, you need to use a CXX that is reasonably similar to the CC.
Otherwise, the CXX will complain about things like attributes not
being supported etc.  That's not surprising, but it's a support issue
that we'll have to prepare ourselves for.

Using my standard set of CC=gcc-7 and CXX=g++-7, the build fails with

g++-7: error: unrecognized command line option '-stdlib=libc++'

That comes from llvm-config --cxxflags, which was apparently made for
/usr/bin/cc (which is clang).

I see here the same problems as we had in the olden days with Perl,
where it gave us a bunch of compiler flags that applied to the system
compiler but not the compiler currently in use.  We should just take the
flags that we really need, like -I and -L.  Maybe we don't need it at all.

Trying it again then with CC=/usr/bin/cc and CXX=/usr/bin/c++, it
fails with

In file included from llvmjit_inline.cpp:25:
In file included from ../../../../src/include/jit/llvmjit.h:16:
In file included from
/usr/local/Cellar/llvm/5.0.1/include/llvm-c/Types.h:17:
In file included from
/usr/local/Cellar/llvm/5.0.1/include/llvm/Support/DataTypes.h:33:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/cmath:555:1:
error: templates must have C++ linkage

Using this patch gets it past that:

diff --git a/src/backend/jit/llvm/llvmjit_inline.cpp
b/src/backend/jit/llvm/llvmjit_inline.cpp
index d4204d2cd2..ad87cfd2d9 100644
--- a/src/backend/jit/llvm/llvmjit_inline.cpp
+++ b/src/backend/jit/llvm/llvmjit_inline.cpp
@@ -22,7 +22,6 @@
 extern "C"
 {
 #include "postgres.h"
-#include "jit/llvmjit.h"

 #include <fcntl.h>
 #include <sys/mman.h>
@@ -35,6 +34,8 @@ extern "C"
 #include "storage/fd.h"
 }

+#include "jit/llvmjit.h"
+
 #include <llvm-c/Core.h>
 #include <llvm-c/BitReader.h>

It seems that it was intended that way anyway, since llvmjit.h contains
its own provisions for extern C.

Then, I'm getting this error:

/usr/bin/cc -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-Wno-unused-command-line-argument -g -O2 -Wno-deprecated-declarations
-Werror   -bundle -multiply_defined suppress -o llvmjit.so  llvmjit.o
llvmjit_error.o llvmjit_inline.o llvmjit_wrap.o llvmjit_expr.o
llvmjit_deform.o -L../../../../src/port -L../../../../src/common
-L/usr/local/lib -L/usr/local/opt/openssl/lib
-L/usr/local/opt/readline/lib -L/usr/local/Cellar/libxml2/2.9.7/lib
-L/usr/local/Cellar/llvm/5.0.1/lib  -L/usr/local/lib
-L/usr/local/opt/openssl/lib -L/usr/local/opt/readline/lib
-Wl,-dead_strip_dylibs  -Werror  -lLLVMPasses -lLLVMipo
-lLLVMInstrumentation -lLLVMVectorize -lLLVMLinker -lLLVMIRReader
-lLLVMAsmParser -lLLVMOrcJIT -lLLVMX86Disassembler -lLLVMX86AsmParser
-lLLVMX86CodeGen -lLLVMGlobalISel -lLLVMSelectionDAG -lLLVMAsmPrinter
-lLLVMDebugInfoCodeView -lLLVMDebugInfoMSF -lLLVMCodeGen
-lLLVMScalarOpts -lLLVMInstCombine -lLLVMTransformUtils -lLLVMBitWriter
-lLLVMX86Desc -lLLVMMCDisassembler -lLLVMX86Info -lLLVMX86AsmPrinter
-lLLVMX86Utils -lLLVMMCJIT -lLLVMExecutionEngine -lLLVMTarget
-lLLVMAnalysis -lLLVMProfileData -lLLVMRuntimeDyld -lLLVMDebugInfoDWARF
-lLLVMObject -lLLVMMCParser -lLLVMMC -lLLVMBitReader -lLLVMCore
-lLLVMBinaryFormat -lLLVMSupport -lLLVMDemangle -lcurses -lz -lm
-bundle_loader ../../../../src/backend/postgres
Undefined symbols for architecture x86_64:
  "std::__1::error_code::message() const", referenced from:
      _LLVMPrintModuleToFile in libLLVMCore.a(Core.cpp.o)
      _LLVMCreateMemoryBufferWithContentsOfFile in libLLVMCore.a(Core.cpp.o)
      _LLVMCreateMemoryBufferWithSTDIN in libLLVMCore.a(Core.cpp.o)
      _LLVMTargetMachineEmitToFile in libLLVMTarget.a(TargetMachineC.cpp.o)
      llvm::errorToErrorCode(llvm::Error) in libLLVMSupport.a(Error.cpp.o)
      llvm::ECError::log(llvm::raw_ostream&) const in
libLLVMSupport.a(Error.cpp.o)

llvm::SectionMemoryManager::finalizeMemory(std::__1::basic_string<char,
std::__1::char_traits<char>, std::__1::allocator<char> >*) in
libLLVMExecutionEngine.a(SectionMemoryManager.cpp.o)
[snipped about 900 lines]

It seems the problem here is linking C++ with the C compiler.  If I
hack in to use c++ in the above command, it continues, and the build
completes.

configure didn't find any of the LLVMOrc* symbols it was looking for.
Is that a problem?  They seem to be for some debugging support.


So, how do I turn this on then?  I see a bunch of new GUC settings
that are all off by default.  Which ones turn the feature(s) on?

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services