Hi,
On 2020-10-03 18:30:46 -0500, Justin Pryzby wrote:
> On Sat, Oct 03, 2020 at 04:01:49PM -0700, Andres Freund wrote:
> > > I was able to make a small, apparent leak like so. This applies to
> > > postgis3.0/postgres12/LLVM5/centos7, and
> > > postgis3.1alpha/postgres13/LLVM9/centos8.
> > >
> > > Inlining is essential to see the leak, optimization is not. Showing every 9th
> > > query:
> > >
> > > $ for a in `seq 1 99`; do echo "SET jit_inline_above_cost=1; SET jit_optimize_above_cost=-1; SET
jit_above_cost=0;SET jit_expressions=on; SET log_executor_stats=on; SET client_min_messages=debug; SET jit=on; explain
analyzeSELECT st_x(site_geo) FROM sites WHERE site_geo IS NOT NULL;"; done |psql ts 2>&1 |grep 'resident' |sed -n
'1~9p'
> > > ! 46024 kB max resident size
> > > ! 46492 kB max resident size
> >
> > Huh, that's certainly not good. Looking.
>
> Reproduce like:
> postgres=# CREATE EXTENSION postgis;
> postgres=# CREATE TABLE geo(g geometry(point));
> postgres=# INSERT INTO geo SELECT st_GeometryFromText('POINT(11 12)',0) FROM generate_series(1,99999);
> postgres=# SET jit_above_cost=0; SET jit_inline_above_cost=0; SET client_min_messages=debug; SET
log_executor_stats=on;explain analyze SELECT st_x(g) FROM geo;
It's an issue inside LLVM. I'm not yet sure how to avoid it. Basically
whenever we load an IR module to inline from, it renames its type names
to make them not conflict with already known type names in the current
LLVMContext. That, over time, leads to a large number of type names in
memory. We can avoid the re-loading of the module by instead cloning it,
but that still leads to new types being created for some internal
reasons. It's possible to create/drop separate LLVMContext instances,
but we'd have to figure out when to do so - it's too expensive to always
do that.
Greetings,
Andres Freund