Обсуждение: LLVM JITLink attempt II (WIP)

Hi,

Here's a short unfinished but semi-working patch to try out JITLink
instead of RuntimeDyld, since Greg expressed an interest in it.  I'm
not actively working on this myself right now, but in case it helps, I
think the main things to do next are probably: try out the recently
added debugger and perf plugins (I hope these are the equivalents of
the JITEventListener stuff we use), figure out how and when we can
actually cut over, tidy/refactor and possibly consider upstreaming
some C wrappers if we want to minimise the C++ glue we carry.  See
patch for a few more details.

While rebasing, I made it use JITLink for LLVM >= 19, but that's
arbitrary.  Tested on FreeBSD/amd64 + LLVM 19, 20, 21.  I haven't
tried other architectures or 32 bit, and support is clearly moving at
very different speeds with 32 bit support far behind.  There is a
published table[1], but it may already be out of date and doesn't talk
about releases.  Ideally I guess we'd put this change off as long as
possible and then cut over everywhere with a simple version threshold
just as RuntimeDyld is finally removed (which hopefully requires full
feature parity to happen first!), but the reality might be messier:
RISC-V doesn't work with RuntimeDyld[2] and it's reasonable to want
that to work sooner, and it might also be nice to drop the ugly ARM
workaround from commit 9044fc1d as soon as LLVM < 15 falls off our
radar.

With LLVM 22 (bleeding edge main branch) it builds and runs simple
things, but I get a SIGBUS crash in the regression tests even on
unpatched master, something to look into separately...

[1] https://llvm.org/docs/JITLink.html (bottom of page)
[2] https://www.postgresql.org/message-id/flat/20220829074622.2474104-1-alex.fan.q%40gmail.com

On Thu, Sep 18, 2025 at 5:05 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> Here's a short unfinished but semi-working patch to try out JITLink
> instead of RuntimeDyld, since Greg expressed an interest in it.  I'm
> not actively working on this myself right now, but in case it helps, I
> think the main things to do next are probably: try out the recently
> added debugger and perf plugins (I hope these are the equivalents of
> the JITEventListener stuff we use), figure out how and when we can
> actually cut over, tidy/refactor and possibly consider upstreaming
> some C wrappers if we want to minimise the C++ glue we carry.  See
> patch for a few more details.

LLVMOrcCreateObjectLinkingLayerWithInProcessMemoryManager() has been
upstreamed into LLVM 22[1], so this patch became shorter.

We could optionally photocopy that function into our llvm_wrap.cpp
file if we have a need to use JITLink with older LLVM versions.  The
motivations for that might be that it is needed for RISC-V and we want
that to work on current Debian et al without waiting ages, or that we
want to kill off our llvm::backport::SectionMemoryManager sooner
without having to wait for LLVM 22 (which has finally fixed the
underlying problem[2]) to be our minimum version.  Someone said this
is also needed for Windows, and I read somewhere that Visual Studio
ships a copy of LLVM these days and it could be a few releases behind,
so that might also provide a motivation, IDK.

I still don't know how to enable GDB and Perf support with JITLink...

> With LLVM 22 (bleeding edge main branch) it builds and runs simple
> things, but I get a SIGBUS crash in the regression tests even on
> unpatched master, something to look into separately...

That turned out to be not our bug, and has been fixed.

[1] https://github.com/llvm/llvm-project/commit/2222cfe7e11ff3e0434bc696856629199ef0da7c
[2]
https://www.postgresql.org/message-id/flat/CA%2BhUKGJTumad75o8Zao-LFseEbt%3DenbUFCM7LZVV%3Dc8yg2i7dg%40mail.gmail.com

On Sat, Jan 3, 2026 at 4:09 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> I still don't know how to enable GDB and Perf support with JITLink...

jit_debugging_support may just be just a matter of calling
LLVMOrcLLJITEnableDebugSupport()[1], though it looks like it might
only work for ELF (the other interesting object format at this stage
being macOS's MachO, and in the past/possible future AIX's COFF/XCOFF
and eventually Windows COFF/PE), but I'm not sure.  You can tell if
it's working by making it crash/abort in JITed code and seeing if the
stack is decodable by GDB  I had been thinking we were going to have
to write that function because I'd missed its arrival in LLVM 18[2],
d'uh.  It sounds like we might need to wait for or write a similar
function for perf support.  Sorry for drip-feeding incomplete work,
but since others expressed an interest in helping :-)

[1] https://github.com/llvm/llvm-project/issues/174305#issuecomment-3709546309
[2] https://github.com/llvm/llvm-project/commit/54397f9ac128568838f2ac7bfc8e1f94b3eb264d

Hi,

On Tue, Jan 6, 2026 at 1:09 AM Thomas Munro <thomas.munro@gmail.com> wrote:
> jit_debugging_support may just be just a matter of calling
> LLVMOrcLLJITEnableDebugSupport()[1], though it looks like it might
> only work for ELF (the other interesting object format at this stage
> being macOS's MachO, and in the past/possible future AIX's COFF/XCOFF
> and eventually Windows COFF/PE), but I'm not sure.  You can tell if
> it's working by making it crash/abort in JITed code and seeing if the
> stack is decodable by GDB  I had been thinking we were going to have
> to write that function because I'd missed its arrival in LLVM 18[2],
> d'uh.

I've tested the patch while adding the call to
'LLVMOrcLLJITEnableDebugSupport(lljit);', and it works. I have full
backtraces with JIT function names:
  * frame #0: 0x0000f4196f94a254 JIT(0xf4196f956000)`deform_1_2 + 28
    frame #1: 0x0000f4196f94a14c JIT(0xf4196f956000)`evalexpr_1_1 + 172
    frame #2: 0x0000bb2f132f3098
postgres`ExecEvalExprNoReturn(econtext=<unavailable>,
state=<unavailable>) at executor.h:423:13
    frame #3: 0x0000bb2f132f3094
postgres`ExecEvalExprNoReturnSwitchContext(econtext=<unavailable>,
state=<unavailable>) at executor.h:464:2

> It sounds like we might need to wait for or write a similar
> function for perf support.

Yeah, there's no C api provided by LLVM yet. I've just opened a
tentative PR[1] to add it. With the modified LLVM and postgres calling
LLVMOrcLLJITEnablePerfSupport, I see the generated perf dumps in
~/.debug/jit/llvm-IR-jit-20260108-61640f/jit-38489.dump.
Though if I try to perf inject the dumps, I have the following errors
  jitdump file contains invalid or unsupported flags 0xf5880666c26c
  0x2b750 [0xa8]: failed to process type: 10 [Operation not permitted]
But that might be an issue with my setup, I would need to test it
further to understand the issue...

[1] https://github.com/llvm/llvm-project/pull/174973

> Though if I try to perf inject the dumps, I have the following errors
>   jitdump file contains invalid or unsupported flags 0xf5880666c26c
>   0x2b750 [0xa8]: failed to process type: 10 [Operation not permitted]
> But that might be an issue with my setup, I would need to test it
> further to understand the issue...

After some additional investigation, I think that's a LLVM issue: The
Flags field is never initialised so the JIT perf plugin was writing
random data in the JIT dump header.
I've opened a PR[0] to fix this.

[0] https://github.com/llvm/llvm-project/pull/175204