Обсуждение: Add --with-llvm to the development snapshot build of Pg11

Re: jean.pierre.pelletier0 2018-04-01 <5ac02d8d.076f6b0a.49022.565d@mx.google.com>
> I'm testing Postgresql 11 on debian 9.4 stable using development snapshots (stretch-pgdg-testing), and would like to
testthe new jit support but pg_config shows that --with-llvm was not used when building.  Any idea if that would be
activatedat one point ?
 
> I've tried installing clang-3.9 and then reinstalled the same postgresql-11 binary package but from what I can see,
that'snot enough to get jit working in Postgres.
 

Hi Jean-Pierre,

the postgresql-11 package in git is actually already using
--with-llvm, but I had disabled it for stretch-pgdg (and
trusty/xenial) because the default clang and llvm in there are only
version 3.8.

I had not realized that stretch provides 3.9 as well, so the next
package build will use that. Yesterday's build attempt failed for some
other reason, I'll see if I can fix that today.

Thanks for the notice,
Christoph

Re: jean.pierre.pelletier0 2018-04-01 <5ac02d8d.076f6b0a.49022.565d@mx.google.com>
> I'm testing Postgresql 11 on debian 9.4 stable using development snapshots (stretch-pgdg-testing), and would like to test the new jit support but pg_config shows that --with-llvm was not used when building.  Any idea if that would be activated at one point ?
> I've tried installing clang-3.9 and then reinstalled the same postgresql-11 binary package but from what I can see, that's not enough to get jit working in Postgres.

Hi Jean-Pierre,

the postgresql-11 package in git is actually already using
--with-llvm, but I had disabled it for stretch-pgdg (and
trusty/xenial) because the default clang and llvm in there are only
version 3.8.

I had not realized that stretch provides 3.9 as well, so the next
package build will use that. Yesterday's build attempt failed for some
other reason, I'll see if I can fix that today.

Thanks for the notice,
Christoph

Re: jean.pierre.pelletier0 2018-04-02 <5ac236d0.8405c80a.9d062.24a0@mx.google.com>
> Hi Christoph,
> I tried the new package today and jit / llvm works fine in Pg11.
> Thanks for fixing this.

Aye, looks good now. Again, thanks for spotting!

Christoph

Hi,

On 2018-04-02 16:25:07 +0200, Christoph Berg wrote:
> Re: jean.pierre.pelletier0 2018-04-02 <5ac236d0.8405c80a.9d062.24a0@mx.google.com>
> > Hi Christoph,
> > I tried the new package today and jit / llvm works fine in Pg11.
> > Thanks for fixing this.
> 
> Aye, looks good now. Again, thanks for spotting!

Are you planning to put the JIT related stuff in a separate package?
It's setup so you can put
pkglibdir/{llvmjit_types.bc,llvmjit.so,bitcode/} in a separate package,
and not have a dependency to LLVM from the main server package.

Greetings,

Andres Freund

Re: Andres Freund 2018-04-02 <20180402180943.7s74bs5klz55e3lv@alap3.anarazel.de>
> Are you planning to put the JIT related stuff in a separate package?
> It's setup so you can put
> pkglibdir/{llvmjit_types.bc,llvmjit.so,bitcode/} in a separate package,
> and not have a dependency to LLVM from the main server package.

I was pondering eventually doing that, yes. So far it's directly in
the postgresql-11 package.

(The worse bloat is actually in postgresql-server-dev-11 which now
also needs to depend on clang and llvm-dev.)

Christoph

Hi,

On 2018-04-02 20:12:11 +0200, Christoph Berg wrote:
> Re: Andres Freund 2018-04-02 <20180402180943.7s74bs5klz55e3lv@alap3.anarazel.de>
> > Are you planning to put the JIT related stuff in a separate package?
> > It's setup so you can put
> > pkglibdir/{llvmjit_types.bc,llvmjit.so,bitcode/} in a separate package,
> > and not have a dependency to LLVM from the main server package.
> 
> I was pondering eventually doing that, yes. So far it's directly in
> the postgresql-11 package.

Ok.

> (The worse bloat is actually in postgresql-server-dev-11 which now
> also needs to depend on clang and llvm-dev.)

I'm a lot less concerned about that, personally. If you're installing
server headers you're already depending on a lot of other libraries, and
you're not that likely to do so on production systems.

Hm, I guess there'd be build failures of extensions if llvm-$ver and
clang aren't installed, because bitcode files will be automatically
built. I don't think there's a dependency on llvm-$ver-dev however?

Sizewise, the bitcode files for postgres and its extensions aren't that
small. ~10MB compressed or such?

Greetings,

Andres Freund

Re: Andres Freund 2018-04-02 <20180402182419.5bbevkbbscsjvw3s@alap3.anarazel.de>
> > (The worse bloat is actually in postgresql-server-dev-11 which now
> > also needs to depend on clang and llvm-dev.)
> 
> I'm a lot less concerned about that, personally. If you're installing
> server headers you're already depending on a lot of other libraries, and
> you're not that likely to do so on production systems.

Actually not that many - the libs are rather small (at least before
the dependency on libicu-dev arrived), and more importantly, during
the libssl-dev/libssl1.0-dev unentangling during the stretch freeze, I
removed all the dependencies from libpq-dev so it doesn't
automatically pull in ssl, kerberos, icu, etc. (Which doesn't seem to
be a problem for most users, there have been only very few
complaints.)

libllvm4.0 needs 50 MB installed, postgresql-11 needs another 50 MB on
top of it.

postgresql-server-dev-11 is much fatter. Given postgresql-11 is
already installed (including libllvm4.0):

Die folgenden NEUEN Pakete werden installiert:
  binfmt-support clang clang-4.0 libclang-common-4.0-dev libclang1-4.0 libffi-dev libjsoncpp1
  libobjc-7-dev libobjc4 libtinfo-dev llvm llvm-4.0 llvm-4.0-dev llvm-4.0-runtime llvm-dev
  llvm-runtime postgresql-server-dev-11
0 aktualisiert, 17 neu installiert, 0 zu entfernen und 4 nicht aktualisiert.
Es müssen 42,4 MB an Archiven heruntergeladen werden.
Nach dieser Operation werden 255 MB Plattenplatz zusätzlich benutzt.

(I'm not too much concerned either.)

> Hm, I guess there'd be build failures of extensions if llvm-$ver and
> clang aren't installed, because bitcode files will be automatically
> built. I don't think there's a dependency on llvm-$ver-dev however?

Afaict all extensions are (or were) failing now, yes. I have to
investigate llvm-dev vs. llvm, thanks for the pointer.

> Sizewise, the bitcode files for postgres and its extensions aren't that
> small. ~10MB compressed or such?

The Installed-Size of postgresql-11 went up from 22 MB to 42 MB, or
the like. The bigger increase is in number of files, unpacking the
.deb takes much longer now. Of 1517 files in postgresql-11, 968 are
bitcode files.

Christoph