Обсуждение: LLVM 22

Hi,

Ideally we should have all changes for LLVM 22 in our February minor
releases.  I have written up some notes on release synchronisation on
the wiki[1] to show the scheduling problem if we don't.  The second
patch here still needs some validation.

1.  We won't need our local llvm::backport::SectionMemoryManager for
LLVM 22, so it will be nice to draw a line under that messy business.
See commit message for details.

You can review the differences between our in-tree copy and the code
that was finally committed and will shortly ship in LLVM 22 like this:

LLVM_BRANCH=main
LLVM_URL=https://raw.githubusercontent.com/llvm/llvm-project/refs/heads

curl -s \
  $LLVM_URL/$LLVM_BRANCH/llvm/include/llvm/ExecutionEngine/SectionMemoryManager.h
| \
  diff -u - src/include/jit/SectionMemoryManager.h

curl -s \
  $LLVM_URL/$LLVM_BRANCH/llvm/lib/ExecutionEngine/SectionMemoryManager.cpp | \
  diff -u - src/backend/jit/llvm/SectionMemoryManager.cpp

In a week or two, LLVM_BRANCH=release/22.x should work too.  I've
attached the output, which shows the expected changes in our copy,
namely:

* top-of-file comments
* namespace change
* tweaks for older LLVM versions
* tree-wide spellchecks and #include "" -> <> changes

They haven't made any changes on their side, except for adding some
LLVM_ABI macros added in LLVM 20 that we missed.  See commit message
for why we don't want those.

The place in llvmjit_backport.h that does:

-#if defined(__aarch64__)
+#if defined(__aarch64__) && LLVM_VERSION_MAJOR < 22
 #define USE_LLVM_BACKPORT_SECTION_MEMORY_MANAGER

... would be like this in REL_17_STABLE and earlier:

+#if defined(__aarch64__) && LLVM_VERSION_MAJOR > 11 && LLVM_VERSION_MAJOR < 22

That's because we never made the backport work with LLVM < 12, and I
have heard no complaints about that so at this point it looks like we
got away with it.

2.  LLVM 22 changed the semantics of the "lifetime.end" instruction.
See commit message for references.  Without this change, LLVM main/22
assertions fail in the regression tests with messages like this in
postmaster.log:

Intrinsic has incorrect argument type!
ptr @llvm.lifetime.end.p0
Intrinsic has incorrect argument type!
ptr @llvm.lifetime.end.p0
2026-01-02 17:28:31.394 NZDT client backend[42798] pg_regress/boolean
FATAL:  fatal llvm error: Broken module found, compilation aborted!

I haven't seen anything bad happen in non-assertion builds.

Here's a potential minimal fix.  I haven't yet proven that the
optimisation is still working as expected.  Probably need to compile
an expression that calls an inlined function and then a non-inlined
function with jit_dump_bitcode=true, then find the right XXX.bc file
under pgdata, llvm-dis XXX.bc, llc XXX.ll, then visually inspect XXX.s
with enough caffeine to confirm that it's not spilling something (ie
store instructions) where previously it didn't, but I wanted to post
what I had so far to see if anyone has a better idea or an easy way to
test it...

[1] https://wiki.postgresql.org/wiki/LLVM#Cadence

On Sat, Jan 3, 2026 at 3:02 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> 1.  We won't need our local llvm::backport::SectionMemoryManager for
> LLVM 22, so it will be nice to draw a line under that messy business.
> See commit message for details.

While that's true, there is a problem with the patch I posted:
"ReserveAlloc" is not enabled when called from C.  I can't actually
reproduce the issue locally due to lack of RAM connected to an ARM
CPU, or I'd have noticed that...  I'll attempt to do something about
that upstream[1], let's see...  if not, we can still use the new
in-tree SectionMemoryManager, but we'll still need some C++ glue code.

[1] https://github.com/llvm/llvm-project/issues/174305

Hi,

On Fri Jan 2, 2026 at 11:02 PM -03, Thomas Munro wrote:
> 2.  LLVM 22 changed the semantics of the "lifetime.end" instruction.
> See commit message for references.  Without this change, LLVM main/22
> assertions fail in the regression tests with messages like this in
> postmaster.log:
>
> Intrinsic has incorrect argument type!
> ptr @llvm.lifetime.end.p0
> Intrinsic has incorrect argument type!
> ptr @llvm.lifetime.end.p0
> 2026-01-02 17:28:31.394 NZDT client backend[42798] pg_regress/boolean
> FATAL:  fatal llvm error: Broken module found, compilation aborted!
>
I've managed to reproduce this using LLVM 22.

> Here's a potential minimal fix.  I haven't yet proven that the
> optimisation is still working as expected.  Probably need to compile
> an expression that calls an inlined function and then a non-inlined
> function with jit_dump_bitcode=true, then find the right XXX.bc file
> under pgdata, llvm-dis XXX.bc, llc XXX.ll, then visually inspect XXX.s
> with enough caffeine to confirm that it's not spilling something (ie
> store instructions) where previously it didn't, but I wanted to post
> what I had so far to see if anyone has a better idea or an easy way to
> test it...
>
I'm not super familiar with reading assembly code but I tried my best to
inspect the LLVM 22 and LLVM 21 outputs and if I understood correctly I
think that 0002 is working as expected.

I've noticed a reduction on some instructions when using LLVM 22 with
the 0002 patch compared with LLVM 21. For example, here we needed less
instructions to set up the registers:

LLVM 22:
    LBB2_8:                                 ; %b.op.1.start
        mov    x20, #40824                     ; =0x9f78
        movk    x20, #19456, lsl #16
        movk    x20, #1, lsl #32
        ldr    x8, [x23]
        ldrb    w9, [x24]
        str    x8, [x20, #152]
        strb    w9, [x20, #160]

LLVM 21:
    LBB2_8:                                 ; %b.op.1.start
        mov    x25, #25352                     ; =0x6308
        movk    x25, #2946, lsl #16
        movk    x25, #1, lsl #32
        mov    x20, #23533                     ; =0x5bed
        movk    x20, #2946, lsl #16
        movk    x20, #1, lsl #32
        ldr    x8, [x23]
        ldrb    w9, [x24]
        stur    x8, [x25, #-248]
        sturb    w9, [x25, #-240]

I've also noticed that the generated assembly code for LLVM 22 use the
str and strb instructions instead of stur and sturb in some cases, which
according to IA is an improvement but unfortunately I did not find any
reference to prove this, sorry.

To test this I did the following steps:
set jit_above_cost = 0;
set jit_inline_above_cost = 0;
set jit_optimize_above_cost = 0;
set jit_dump_bitcode = true;

explain(analyze) select i % 2 = 0 OR i % 3 = 0 from generate_series(1, 100) i;

I'm attaching the .s files for the llvm 22 and for the llvm 21 outputs
that I used to inspect.

--
Matheus Alcantara
EDB: https://www.enterprisedb.com

On Tue, Jan 6, 2026 at 10:56 AM Matheus Alcantara
<matheusssilv97@gmail.com> wrote:
> On Fri Jan 2, 2026 at 11:02 PM -03, Thomas Munro wrote:
> > Intrinsic has incorrect argument type!
> > ptr @llvm.lifetime.end.p0
> > Intrinsic has incorrect argument type!
> > ptr @llvm.lifetime.end.p0
> > 2026-01-02 17:28:31.394 NZDT client backend[42798] pg_regress/boolean
> > FATAL:  fatal llvm error: Broken module found, compilation aborted!
> >
> I've managed to reproduce this using LLVM 22.

Thanks for testing!

> > Here's a potential minimal fix.  I haven't yet proven that the
> > optimisation is still working as expected.  Probably need to compile
> > an expression that calls an inlined function and then a non-inlined
> > function with jit_dump_bitcode=true, then find the right XXX.bc file
> > under pgdata, llvm-dis XXX.bc, llc XXX.ll, then visually inspect XXX.s
> > with enough caffeine to confirm that it's not spilling something (ie
> > store instructions) where previously it didn't, but I wanted to post
> > what I had so far to see if anyone has a better idea or an easy way to
> > test it...
> >
> I'm not super familiar with reading assembly code but I tried my best to
> inspect the LLVM 22 and LLVM 21 outputs and if I understood correctly I
> think that 0002 is working as expected.

Cool.  And as another sanity test, if you comment out the new poison
code so that we don't try to prevent unwanted spills/stores, can you
see any?

> I've noticed a reduction on some instructions when using LLVM 22 with
> the 0002 patch compared with LLVM 21. For example, here we needed less
> instructions to set up the registers:
>
> LLVM 22:
>     LBB2_8:                                 ; %b.op.1.start
>         mov     x20, #40824                     ; =0x9f78
>         movk    x20, #19456, lsl #16
>         movk    x20, #1, lsl #32
>         ldr     x8, [x23]
>         ldrb    w9, [x24]
>         str     x8, [x20, #152]
>         strb    w9, [x20, #160]
>
> LLVM 21:
>     LBB2_8:                                 ; %b.op.1.start
>         mov     x25, #25352                     ; =0x6308
>         movk    x25, #2946, lsl #16
>         movk    x25, #1, lsl #32
>         mov     x20, #23533                     ; =0x5bed
>         movk    x20, #2946, lsl #16
>         movk    x20, #1, lsl #32
>         ldr     x8, [x23]
>         ldrb    w9, [x24]
>         stur    x8, [x25, #-248]
>         sturb   w9, [x25, #-240]
>
> I've also noticed that the generated assembly code for LLVM 22 use the
> str and strb instructions instead of stur and sturb in some cases, which
> according to IA is an improvement but unfortunately I did not find any
> reference to prove this, sorry.

Interesting.

On Mon Jan 5, 2026 at 8:50 PM -03, Thomas Munro wrote:
>> > Here's a potential minimal fix.  I haven't yet proven that the
>> > optimisation is still working as expected.  Probably need to compile
>> > an expression that calls an inlined function and then a non-inlined
>> > function with jit_dump_bitcode=true, then find the right XXX.bc file
>> > under pgdata, llvm-dis XXX.bc, llc XXX.ll, then visually inspect XXX.s
>> > with enough caffeine to confirm that it's not spilling something (ie
>> > store instructions) where previously it didn't, but I wanted to post
>> > what I had so far to see if anyone has a better idea or an easy way to
>> > test it...
>> >
>> I'm not super familiar with reading assembly code but I tried my best to
>> inspect the LLVM 22 and LLVM 21 outputs and if I understood correctly I
>> think that 0002 is working as expected.
>
> Cool.  And as another sanity test, if you comment out the new poison
> code so that we don't try to prevent unwanted spills/stores, can you
> see any?
>
Yes, I've commented the poison code block introduced on 0002 and the
generated assembly code seems more bloated, for example:

LLVM 22 with 0002 and the poison code block commented:
    LBB2_8:
        mov    x25, #15624                     ; =0x3d08
        movk    x25, #7427, lsl #16
        movk    x25, #1, lsl #32
        mov    x20, #13805                     ; =0x35ed
        movk    x20, #7427, lsl #16
        movk    x20, #1, lsl #32
        ldr    x8, [x23]
        ldrb    w9, [x24]
        stur    x8, [x25, #-248]
        sturb    w9, [x25, #-240]
        mov    w26, #1                         ; =0x1
        strb    w26, [x20, #1419]
        ldurb    w8, [x25, #-240]
        cmp    w8, #1
        b.eq    LBB2_11

LLVM 22 with 0002:
    LBB2_8:
        mov    x20, #40824                     ; =0x9f78
        movk    x20, #19456, lsl #16
        movk    x20, #1, lsl #32
        ldr    x8, [x23]
        ldrb    w9, [x24]
        str    x8, [x20, #152]
        strb    w9, [x20, #160]
        mov    w26, #1                         ; =0x1
        strb    w26, [x20]
        cmp    w9, #1
        b.eq    LBB2_11

IIUC with the commented code the LLVM compiler added an extra
load ldurb followed by cmp w8, #1. With the patch it performs a
comparison cmp w9, #1 directly using a register it already has.

--
Matheus Alcantara
EDB: https://www.enterprisedb.com