Обсуждение: ERROR: XX000: could not find memoization table entry (reproducible)

​memo.dump​

Emmanuel Touzery Razvoj programske opreme E: emmanuel.touzery@plandela.si

PLANDELA d.o.o. Cesta Ljubljanske brigade 23a 1000 Ljubljana www.plandela.si

To sporočilo in morebitne priloge so poslovna skrivnost in namenjene izključno naslovniku, če ni navedeno drugače. Prosimo, da vsebino sporočila in prilog varujete v skladu z veljavno zakonodajo in pogodbenimi obveznostmi. Če ste sporočilo prejeli pomotoma, vas prosimo, da takoj obvestite pošiljatelja, sporočilo in priloge pa nepovratno uničite.

This e-mail and any attachments contain confidential information and is intended solely for the addressee unless stated otherwise. Please uphold strict confidentiality of the content of this message and attachments in accordance with the law and contractual obligations. If you have received this e-mail by mistake, please immediately notify the sender and irreversibly destroy this message and attachments.

On Wed, 22 Oct 2025 at 20:36, Emmanuel Touzery
<emmanuel.touzery@plandela.si> wrote:
> When the issue occurs... With \set verbose, psql displays:
>
> ERROR:  XX000: could not find memoization table entry
> LOCATION:  cache_reduce_memory, nodeMemoize.c:484

Thanks for the recreation steps and dump file. I can recreate it here.
I'm just looking now.

David

On Wed, 22 Oct 2025 at 21:02, David Rowley <dgrowleyml@gmail.com> wrote:
>
> On Wed, 22 Oct 2025 at 20:36, Emmanuel Touzery
> <emmanuel.touzery@plandela.si> wrote:
> > When the issue occurs... With \set verbose, psql displays:
> >
> > ERROR:  XX000: could not find memoization table entry
> > LOCATION:  cache_reduce_memory, nodeMemoize.c:484
>
> Thanks for the recreation steps and dump file. I can recreate it here.
> I'm just looking now.

Are you able to confirm if you still get the error with:

SET jit_tuple_deforming TO off;

and if you do; does it still happen if you do:

SET jit TO off;

David

To sporočilo in morebitne priloge so poslovna skrivnost in namenjene izključno naslovniku, če ni navedeno drugače. Prosimo, da vsebino sporočila in prilog varujete v skladu z veljavno zakonodajo in pogodbenimi obveznostmi. Če ste sporočilo prejeli pomotoma, vas prosimo, da takoj obvestite pošiljatelja, sporočilo in priloge pa nepovratno uničite.

This e-mail and any attachments contain confidential information and is intended solely for the addressee unless stated otherwise. Please uphold strict confidentiality of the content of this message and attachments in accordance with the law and contractual obligations. If you have received this e-mail by mistake, please immediately notify the sender and irreversibly destroy this message and attachments.

On Wed, 22 Oct 2025 at 21:48, Emmanuel Touzery
<emmanuel.touzery@plandela.si> wrote:
> I can confirm that seting jit_tuple_deforming to off is enough to fix the issue.

Thanks for checking that.

It does look like something weird is going on with the JIT deforming
code. I was playing around with the attached patch applied to look at
the tuple data. Rather surprisingly, the Datum value for negative ints
isn't the same for JIT deform vs normal deform.

Here's an example:

test=# set jit_above_cost=0;
SET
test=# create table a (a int);
CREATE TABLE
test=# insert into a values(-353400);
INSERT 0 1
test=# set jit=1;
SET
test=# explain analyze select * from a where a < 0;
NOTICE:  scan tuple = (a) (-353400) (4294613896)

test=# set jit=0;
SET
test=# explain analyze select * from a where a < 0;
NOTICE:  scan tuple = (a) (-353400) (18446744073709198216)

The format there is (<column>) (<logical value>) (<raw datum value>)

The lower 32 bits of these numbers are the same:

0000000000000000000000000000000011111111111110101001101110001000
1111111111111111111111111111111111111111111110101001101110001000

It's the upper 32 bits that are all 1s for no JIT and all zeros for JIT.

The reason this is causing Memoize indigestion is because when in
binary mode, the cache lookup is done with datum_image_hash() and
datum_image_eq(). I didn't check exactly, but I suspect that the value
we get when deforming the cache key's MinimalTuple has the Datum with
the non JIT representation.

To figure out why this is happening requires caffeination and staring
at llvmjit_deform.c for a long time. I suspect whatever is in there
that's meant to do what fetch_att() does isn't doing it the same way.

David

On Thu, 23 Oct 2025 at 00:16, David Rowley <dgrowleyml@gmail.com> wrote:
> To figure out why this is happening requires caffeination and staring
> at llvmjit_deform.c for a long time. I suspect whatever is in there
> that's meant to do what fetch_att() does isn't doing it the same way.

This code here:

LLVMValueRef v_tmp_loaddata;
LLVMTypeRef vartype = LLVMIntTypeInContext(lc, att->attlen * 8);
LLVMTypeRef vartypep = LLVMPointerType(vartype, 0);

v_tmp_loaddata =
LLVMBuildPointerCast(b, v_attdatap, vartypep, "");
v_tmp_loaddata = l_load(b, vartype, v_tmp_loaddata, "attr_byval");
v_tmp_loaddata = LLVMBuildZExt(b, v_tmp_loaddata, TypeDatum, "");

LLVMBuildStore(b, v_tmp_loaddata, v_resultp);

I don't speak this fluently by any means, but I think I see a zero
extension here up to the Datum width. That's not what fetch_att()
does.

Will look more tomorrow.

David

On Thu, 23 Oct 2025 at 00:27, David Rowley <dgrowleyml@gmail.com> wrote:
> v_tmp_loaddata = LLVMBuildZExt(b, v_tmp_loaddata, TypeDatum, "");

> Will look more tomorrow.

This seems to be a 1 liner fix. Basically, the zero extend should be a
signed extend. i.e, use LLVMBuildSExt().

Using [1] to get the extra debug output.

set jit_Above_Cost=0;
drop table if exists b;
create table b (a char, b smallint, c int);
insert into b values('-100'::char,-123,-1234);
insert into b values('123'::char,1234,12345);
set jit=1;
select * from b where c <> 0;
set jit=0;
select * from b where c <> 0;

With the patch, I get:

NOTICE:  scan tuple = (a,b,c) (-,-123,-1234)
(124262785167352,18446744073709551493,18446744073709550382)
NOTICE:  scan tuple = (a,b,c) (1,1234,12345) (124262785167320,1234,12345)
 a |  b   |   c
---+------+-------
 - | -123 | -1234
 1 | 1234 | 12345
(2 rows)

SET
NOTICE:  scan tuple = (a,b,c) (-,-123,-1234)
(124262785167352,18446744073709551493,18446744073709550382)
NOTICE:  scan tuple = (a,b,c) (1,1234,12345) (124262785167320,1234,12345)
 a |  b   |   c
---+------+-------
 - | -123 | -1234
 1 | 1234 | 12345
(2 rows)

And without, I get:

NOTICE:  scan tuple = (a,b,c) (-,-123,-1234) (135890207272952,65413,4294966062)
NOTICE:  scan tuple = (a,b,c) (1,1234,12345) (135890207272920,1234,12345)
 a |  b   |   c
---+------+-------
 - | -123 | -1234
 1 | 1234 | 12345
(2 rows)

SET
NOTICE:  scan tuple = (a,b,c) (-,-123,-1234)
(135890207272952,18446744073709551493,18446744073709550382)
NOTICE:  scan tuple = (a,b,c) (1,1234,12345) (135890207272920,1234,12345)
 a |  b   |   c
---+------+-------
 - | -123 | -1234
 1 | 1234 | 12345
(2 rows)

David

[1] https://www.postgresql.org/message-id/attachment/183414/debug_scan_data.diff

On Thu, 23 Oct 2025 at 10:04, David Rowley <dgrowleyml@gmail.com> wrote:
> This seems to be a 1 liner fix. Basically, the zero extend should be a
> signed extend. i.e, use LLVMBuildSExt().

The patch for the 1-liner.

David

On Thu, 23 Oct 2025 at 10:13, David Rowley <dgrowleyml@gmail.com> wrote:
> The patch for the 1-liner.

I've pushed the fix for this [1]. The only other instance I saw of
LLVMBuildZExt for TypeDatum is in llvmjit_expr.c for "case
EEOP_BOOL_NOT_STEP". I didn't adjust this as it wasn't clear to me
that it was a problem. I expect the value being zero extended there is
either a 0 or 1, so didn't want to risk touching that code if it isn't
broken.


Emmanuel,

You should be seeing it in the next minor releases due around
mid-November. In the meantime, I suggest turning off
jit_tuple_deforming.

Many thanks for reporting this and coming up with the steps and backup
to reproduce the issue.

David

[1] https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=10945148eec689cc114fe872d623c3c7c4f546e6

Emmanuel Touzery Razvoj programske opreme E: emmanuel.touzery@plandela.si

PLANDELA d.o.o. Cesta Ljubljanske brigade 23a 1000 Ljubljana www.plandela.si

To sporočilo in morebitne priloge so poslovna skrivnost in namenjene izključno naslovniku, če ni navedeno drugače. Prosimo, da vsebino sporočila in prilog varujete v skladu z veljavno zakonodajo in pogodbenimi obveznostmi. Če ste sporočilo prejeli pomotoma, vas prosimo, da takoj obvestite pošiljatelja, sporočilo in priloge pa nepovratno uničite.

This e-mail and any attachments contain confidential information and is intended solely for the addressee unless stated otherwise. Please uphold strict confidentiality of the content of this message and attachments in accordance with the law and contractual obligations. If you have received this e-mail by mistake, please immediately notify the sender and irreversibly destroy this message and attachments.

On Thu, 23 Oct 2025 at 19:30, Emmanuel Touzery
<emmanuel.touzery@plandela.si> wrote:
>  Maybe just a followup question: will the fix also be included in upcoming postgresql 17 minor releases? Seeing that
it'snot only PG18 that's affected.
 

Yes, that's the plan. All the way back to PostgreSQL 13.

David