Обсуждение: PostgreSQL 12, JIT defaults

Поиск
Список
Период
Сортировка

PostgreSQL 12, JIT defaults

От
Pavel Stehule
Дата:
Hi

I configured PostgreSQL without JIT support, but JIT is active by default

postgres=# show jit;
┌─────┐
│ jit │
╞═════╡
│ on  │
└─────┘
(1 row)

Unfortunately - this combination does crashes on some bigger queries.

Regards

Pavel

Re: PostgreSQL 12, JIT defaults

От
Thomas Munro
Дата:
On Mon, Oct 8, 2018 at 10:52 PM Pavel Stehule <pavel.stehule@gmail.com> wrote:
>
> Hi
>
> I configured PostgreSQL without JIT support, but JIT is active by default

I think that happens when llvmjit.so is present (ie from last time you
built with JIT support and ran make install).  You need to remove it.

--
Thomas Munro
http://www.enterprisedb.com


Re: PostgreSQL 12, JIT defaults

От
Pavel Stehule
Дата:


po 8. 10. 2018 v 12:06 odesílatel Thomas Munro <thomas.munro@enterprisedb.com> napsal:
On Mon, Oct 8, 2018 at 10:52 PM Pavel Stehule <pavel.stehule@gmail.com> wrote:
>
> Hi
>
> I configured PostgreSQL without JIT support, but JIT is active by default

I think that happens when llvmjit.so is present (ie from last time you
built with JIT support and ran make install).  You need to remove it.

aha. I'll do it. But it doesn't look like robust solution :-(.

Maybe clean, or distclean should to remove this file.

Thank you for info

Pavel


--
Thomas Munro
http://www.enterprisedb.com

Re: PostgreSQL 12, JIT defaults

От
Andres Freund
Дата:
Hi

On October 8, 2018 2:51:15 AM PDT, Pavel Stehule <pavel.stehule@gmail.com> wrote:
>Hi
>
>I configured PostgreSQL without JIT support, but JIT is active by
>default
>
>postgres=# show jit;
>┌─────┐
>│ jit │
>╞═════╡
>│ on  │
>└─────┘
>(1 row)

Where is the jit=on coming from? Config from before it was turned off?


>Unfortunately - this combination does crashes on some bigger queries.

You probably have the JIT plugin installed from an earlier time, which also would explain why it crashes.

Andres

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.


Re: PostgreSQL 12, JIT defaults

От
Andres Freund
Дата:
Hi

On October 8, 2018 2:51:15 AM PDT, Pavel Stehule <pavel.stehule@gmail.com> wrote:
>Hi
>
>I configured PostgreSQL without JIT support, but JIT is active by
>default
>
>postgres=# show jit;
>┌─────┐
>│ jit │
>╞═════╡
>│ on  │
>└─────┘
>(1 row)

Where is the jit=on coming from? Config from before it was turned off?


>Unfortunately - this combination does crashes on some bigger queries.

You probably have the JIT plugin installed from an earlier time, which also would explain why it crashes.

Andres

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.



Re: PostgreSQL 12, JIT defaults

От
Tom Lane
Дата:
Andres Freund <andres@anarazel.de> writes:
> Where is the jit=on coming from? Config from before it was turned off?

A look in guc.c shows that jit defaults to "on" whether or not JIT is
enabled at compile time.  This seems, at best, rather user-unfriendly.
And it's not like our conventions for other compile-option-affected
GUCs, eg the SSL ones.

            regards, tom lane



Re: PostgreSQL 12, JIT defaults

От
Tom Lane
Дата:
Andres Freund <andres@anarazel.de> writes:
> Where is the jit=on coming from? Config from before it was turned off?

A look in guc.c shows that jit defaults to "on" whether or not JIT is
enabled at compile time.  This seems, at best, rather user-unfriendly.
And it's not like our conventions for other compile-option-affected
GUCs, eg the SSL ones.

            regards, tom lane


Re: PostgreSQL 12, JIT defaults

От
Pavel Stehule
Дата:


po 8. 10. 2018 v 16:58 odesílatel Andres Freund <andres@anarazel.de> napsal:
Hi

On October 8, 2018 2:51:15 AM PDT, Pavel Stehule <pavel.stehule@gmail.com> wrote:
>Hi
>
>I configured PostgreSQL without JIT support, but JIT is active by
>default
>
>postgres=# show jit;
>┌─────┐
>│ jit │
>╞═════╡
>│ on  │
>└─────┘
>(1 row)

Where is the jit=on coming from? Config from before it was turned off?


>Unfortunately - this combination does crashes on some bigger queries.

You probably have the JIT plugin installed from an earlier time, which also would explain why it crashes.

I had it there.
 

Andres

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: PostgreSQL 12, JIT defaults

От
Pavel Stehule
Дата:


po 8. 10. 2018 v 16:58 odesílatel Andres Freund <andres@anarazel.de> napsal:
Hi

On October 8, 2018 2:51:15 AM PDT, Pavel Stehule <pavel.stehule@gmail.com> wrote:
>Hi
>
>I configured PostgreSQL without JIT support, but JIT is active by
>default
>
>postgres=# show jit;
>┌─────┐
>│ jit │
>╞═════╡
>│ on  │
>└─────┘
>(1 row)

Where is the jit=on coming from? Config from before it was turned off?


>Unfortunately - this combination does crashes on some bigger queries.

You probably have the JIT plugin installed from an earlier time, which also would explain why it crashes.

I had it there.
 

Andres

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: PostgreSQL 12, JIT defaults

От
Andres Freund
Дата:

On October 8, 2018 8:03:56 AM PDT, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>Andres Freund <andres@anarazel.de> writes:
>> Where is the jit=on coming from? Config from before it was turned
>off?
>
>A look in guc.c shows that jit defaults to "on" whether or not JIT is
>enabled at compile time.

I thought Pavel was talking about 11 somehow...


> This seems, at best, rather user-unfriendly.
>And it's not like our conventions for other compile-option-affected
>GUCs, eg the SSL ones.

That was intentional, even though it perhaps should be better documented. That allows a distro to build and distribute
pgwithout llvm enabled, but then have the JIT package with all the dependencies separately. The pg yum packages do so. 

Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.


Re: PostgreSQL 12, JIT defaults

От
Andres Freund
Дата:

On October 8, 2018 8:03:56 AM PDT, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>Andres Freund <andres@anarazel.de> writes:
>> Where is the jit=on coming from? Config from before it was turned
>off?
>
>A look in guc.c shows that jit defaults to "on" whether or not JIT is
>enabled at compile time.

I thought Pavel was talking about 11 somehow...


> This seems, at best, rather user-unfriendly.
>And it's not like our conventions for other compile-option-affected
>GUCs, eg the SSL ones.

That was intentional, even though it perhaps should be better documented. That allows a distro to build and distribute
pgwithout llvm enabled, but then have the JIT package with all the dependencies separately. The pg yum packages do so. 

Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.



Re: PostgreSQL 12, JIT defaults

От
Andres Freund
Дата:

On October 8, 2018 8:10:45 AM PDT, Andres Freund <andres@anarazel.de> wrote:
>
>
>On October 8, 2018 8:03:56 AM PDT, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> This seems, at best, rather user-unfriendly.
>>And it's not like our conventions for other compile-option-affected
>>GUCs, eg the SSL ones.
>
>That was intentional, even though it perhaps should be better
>documented. That allows a distro to build and distribute pg without
>llvm enabled, but then have the JIT package with all the dependencies
>separately. The pg yum packages do so.

There's a function for checking whether JIT is actually available, BTW. pg_jit_available(). That also works if somebody
installsan extension that's not ours (by setting jit_provider =  ...). 


Andres

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.


Re: PostgreSQL 12, JIT defaults

От
Andres Freund
Дата:

On October 8, 2018 8:10:45 AM PDT, Andres Freund <andres@anarazel.de> wrote:
>
>
>On October 8, 2018 8:03:56 AM PDT, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> This seems, at best, rather user-unfriendly.
>>And it's not like our conventions for other compile-option-affected
>>GUCs, eg the SSL ones.
>
>That was intentional, even though it perhaps should be better
>documented. That allows a distro to build and distribute pg without
>llvm enabled, but then have the JIT package with all the dependencies
>separately. The pg yum packages do so.

There's a function for checking whether JIT is actually available, BTW. pg_jit_available(). That also works if somebody
installsan extension that's not ours (by setting jit_provider =  ...). 


Andres

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.



Re: PostgreSQL 12, JIT defaults

От
Pavel Stehule
Дата:


po 8. 10. 2018 v 17:10 odesílatel Andres Freund <andres@anarazel.de> napsal:


On October 8, 2018 8:03:56 AM PDT, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>Andres Freund <andres@anarazel.de> writes:
>> Where is the jit=on coming from? Config from before it was turned
>off?
>
>A look in guc.c shows that jit defaults to "on" whether or not JIT is
>enabled at compile time.

I thought Pavel was talking about 11 somehow...

I am sorry, It was not clear - I talked about master



> This seems, at best, rather user-unfriendly.
>And it's not like our conventions for other compile-option-affected
>GUCs, eg the SSL ones.

That was intentional, even though it perhaps should be better documented. That allows a distro to build and distribute pg without llvm enabled, but then have the JIT package with all the dependencies separately. The pg yum packages do so.

I don't like this solution - it is safe on production - but it can breaks my development environment - or we need to change setup and make install will not be enough.

Regards

Pavel


Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: PostgreSQL 12, JIT defaults

От
Pavel Stehule
Дата:


po 8. 10. 2018 v 17:10 odesílatel Andres Freund <andres@anarazel.de> napsal:


On October 8, 2018 8:03:56 AM PDT, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>Andres Freund <andres@anarazel.de> writes:
>> Where is the jit=on coming from? Config from before it was turned
>off?
>
>A look in guc.c shows that jit defaults to "on" whether or not JIT is
>enabled at compile time.

I thought Pavel was talking about 11 somehow...

I am sorry, It was not clear - I talked about master



> This seems, at best, rather user-unfriendly.
>And it's not like our conventions for other compile-option-affected
>GUCs, eg the SSL ones.

That was intentional, even though it perhaps should be better documented. That allows a distro to build and distribute pg without llvm enabled, but then have the JIT package with all the dependencies separately. The pg yum packages do so.

I don't like this solution - it is safe on production - but it can breaks my development environment - or we need to change setup and make install will not be enough.

Regards

Pavel


Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: PostgreSQL 12, JIT defaults

От
Andres Freund
Дата:

On October 8, 2018 8:16:06 AM PDT, Pavel Stehule <pavel.stehule@gmail.com> wrote:
>po 8. 10. 2018 v 17:10 odesílatel Andres Freund <andres@anarazel.de>
>napsal:
>
>>
>>
>> On October 8, 2018 8:03:56 AM PDT, Tom Lane <tgl@sss.pgh.pa.us>
>wrote:
>> >Andres Freund <andres@anarazel.de> writes:
>> >> Where is the jit=on coming from? Config from before it was turned
>> >off?
>> >
>> >A look in guc.c shows that jit defaults to "on" whether or not JIT
>is
>> >enabled at compile time.
>>
>> I thought Pavel was talking about 11 somehow...
>>
>
>I am sorry, It was not clear - I talked about master
>
>
>>
>> > This seems, at best, rather user-unfriendly.
>> >And it's not like our conventions for other compile-option-affected
>> >GUCs, eg the SSL ones.
>>
>> That was intentional, even though it perhaps should be better
>documented.
>> That allows a distro to build and distribute pg without llvm enabled,
>but
>> then have the JIT package with all the dependencies separately. The
>pg yum
>> packages do so.
>>
>
>I don't like this solution - it is safe on production - but it can
>breaks
>my development environment - or we need to change setup and make
>install
>will not be enough.

It's not clear what could be done about it. You'll run into other trouble if you have half installed build artifacts
becauseyou reconfigured. Make uninstall from before the reconfigure would uninstall it. 

Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.



Re: PostgreSQL 12, JIT defaults

От
Andres Freund
Дата:

On October 8, 2018 8:16:06 AM PDT, Pavel Stehule <pavel.stehule@gmail.com> wrote:
>po 8. 10. 2018 v 17:10 odesílatel Andres Freund <andres@anarazel.de>
>napsal:
>
>>
>>
>> On October 8, 2018 8:03:56 AM PDT, Tom Lane <tgl@sss.pgh.pa.us>
>wrote:
>> >Andres Freund <andres@anarazel.de> writes:
>> >> Where is the jit=on coming from? Config from before it was turned
>> >off?
>> >
>> >A look in guc.c shows that jit defaults to "on" whether or not JIT
>is
>> >enabled at compile time.
>>
>> I thought Pavel was talking about 11 somehow...
>>
>
>I am sorry, It was not clear - I talked about master
>
>
>>
>> > This seems, at best, rather user-unfriendly.
>> >And it's not like our conventions for other compile-option-affected
>> >GUCs, eg the SSL ones.
>>
>> That was intentional, even though it perhaps should be better
>documented.
>> That allows a distro to build and distribute pg without llvm enabled,
>but
>> then have the JIT package with all the dependencies separately. The
>pg yum
>> packages do so.
>>
>
>I don't like this solution - it is safe on production - but it can
>breaks
>my development environment - or we need to change setup and make
>install
>will not be enough.

It's not clear what could be done about it. You'll run into other trouble if you have half installed build artifacts
becauseyou reconfigured. Make uninstall from before the reconfigure would uninstall it. 

Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.


Re: PostgreSQL 12, JIT defaults

От
Pavel Stehule
Дата:


po 8. 10. 2018 v 17:22 odesílatel Andres Freund <andres@anarazel.de> napsal:


On October 8, 2018 8:16:06 AM PDT, Pavel Stehule <pavel.stehule@gmail.com> wrote:
>po 8. 10. 2018 v 17:10 odesílatel Andres Freund <andres@anarazel.de>
>napsal:
>
>>
>>
>> On October 8, 2018 8:03:56 AM PDT, Tom Lane <tgl@sss.pgh.pa.us>
>wrote:
>> >Andres Freund <andres@anarazel.de> writes:
>> >> Where is the jit=on coming from? Config from before it was turned
>> >off?
>> >
>> >A look in guc.c shows that jit defaults to "on" whether or not JIT
>is
>> >enabled at compile time.
>>
>> I thought Pavel was talking about 11 somehow...
>>
>
>I am sorry, It was not clear - I talked about master
>
>
>>
>> > This seems, at best, rather user-unfriendly.
>> >And it's not like our conventions for other compile-option-affected
>> >GUCs, eg the SSL ones.
>>
>> That was intentional, even though it perhaps should be better
>documented.
>> That allows a distro to build and distribute pg without llvm enabled,
>but
>> then have the JIT package with all the dependencies separately. The
>pg yum
>> packages do so.
>>
>
>I don't like this solution - it is safe on production - but it can
>breaks
>my development environment - or we need to change setup and make
>install
>will not be enough.

It's not clear what could be done about it. You'll run into other trouble if you have half installed build artifacts because you reconfigured. Make uninstall from before the reconfigure would uninstall it.

It is partially true - when I use wrong extension, then I got error message. But JIT library kills Postgres.

I expecting permanently disabled JIT if postgres was compiled without JIT support. Dependency on some external file doesn't looks like safe solution.


Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: PostgreSQL 12, JIT defaults

От
Pavel Stehule
Дата:


po 8. 10. 2018 v 17:22 odesílatel Andres Freund <andres@anarazel.de> napsal:


On October 8, 2018 8:16:06 AM PDT, Pavel Stehule <pavel.stehule@gmail.com> wrote:
>po 8. 10. 2018 v 17:10 odesílatel Andres Freund <andres@anarazel.de>
>napsal:
>
>>
>>
>> On October 8, 2018 8:03:56 AM PDT, Tom Lane <tgl@sss.pgh.pa.us>
>wrote:
>> >Andres Freund <andres@anarazel.de> writes:
>> >> Where is the jit=on coming from? Config from before it was turned
>> >off?
>> >
>> >A look in guc.c shows that jit defaults to "on" whether or not JIT
>is
>> >enabled at compile time.
>>
>> I thought Pavel was talking about 11 somehow...
>>
>
>I am sorry, It was not clear - I talked about master
>
>
>>
>> > This seems, at best, rather user-unfriendly.
>> >And it's not like our conventions for other compile-option-affected
>> >GUCs, eg the SSL ones.
>>
>> That was intentional, even though it perhaps should be better
>documented.
>> That allows a distro to build and distribute pg without llvm enabled,
>but
>> then have the JIT package with all the dependencies separately. The
>pg yum
>> packages do so.
>>
>
>I don't like this solution - it is safe on production - but it can
>breaks
>my development environment - or we need to change setup and make
>install
>will not be enough.

It's not clear what could be done about it. You'll run into other trouble if you have half installed build artifacts because you reconfigured. Make uninstall from before the reconfigure would uninstall it.

It is partially true - when I use wrong extension, then I got error message. But JIT library kills Postgres.

I expecting permanently disabled JIT if postgres was compiled without JIT support. Dependency on some external file doesn't looks like safe solution.


Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: PostgreSQL 12, JIT defaults

От
Tom Lane
Дата:
Andres Freund <andres@anarazel.de> writes:
> On October 8, 2018 8:03:56 AM PDT, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> A look in guc.c shows that jit defaults to "on" whether or not JIT is
>> enabled at compile time.
>> This seems, at best, rather user-unfriendly.
>> And it's not like our conventions for other compile-option-affected
>> GUCs, eg the SSL ones.

> That was intentional, even though it perhaps should be better documented. That allows a distro to build and
distributepg without llvm enabled, but then have the JIT package with all the dependencies separately. The pg yum
packagesdo so. 

I'm not following.  A distro wishing to do that would configure
--with-llvm, not without, and then simply split up the build results
so that the JIT stuff is in a separate subpackage.  If you configured
--without-llvm then the resulting core code is (one hopes) entirely
incapable of using JIT, and it'd be better if the GUC settings
reflected that.

            regards, tom lane


Re: PostgreSQL 12, JIT defaults

От
Tom Lane
Дата:
Andres Freund <andres@anarazel.de> writes:
> On October 8, 2018 8:03:56 AM PDT, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> A look in guc.c shows that jit defaults to "on" whether or not JIT is
>> enabled at compile time.
>> This seems, at best, rather user-unfriendly.
>> And it's not like our conventions for other compile-option-affected
>> GUCs, eg the SSL ones.

> That was intentional, even though it perhaps should be better documented. That allows a distro to build and
distributepg without llvm enabled, but then have the JIT package with all the dependencies separately. The pg yum
packagesdo so. 

I'm not following.  A distro wishing to do that would configure
--with-llvm, not without, and then simply split up the build results
so that the JIT stuff is in a separate subpackage.  If you configured
--without-llvm then the resulting core code is (one hopes) entirely
incapable of using JIT, and it'd be better if the GUC settings
reflected that.

            regards, tom lane



Re: PostgreSQL 12, JIT defaults

От
Andres Freund
Дата:
Hi,

On 2018-10-08 11:43:42 -0400, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > On October 8, 2018 8:03:56 AM PDT, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >> A look in guc.c shows that jit defaults to "on" whether or not JIT is
> >> enabled at compile time. 
> >> This seems, at best, rather user-unfriendly.
> >> And it's not like our conventions for other compile-option-affected
> >> GUCs, eg the SSL ones.
> 
> > That was intentional, even though it perhaps should be better documented. That allows a distro to build and
distributepg without llvm enabled, but then have the JIT package with all the dependencies separately. The pg yum
packagesdo so.
 
> 
> I'm not following.  A distro wishing to do that would configure
> --with-llvm, not without, and then simply split up the build results
> so that the JIT stuff is in a separate subpackage.

Well, that'd then leave you with one more state (LLVM configured but not
installed, LLVM configured and installed, LLVM disabled and arguably
LLVM disabled but installed), but more importantly if you compile
without llvm enabled, you can still install a different extension that
would do JIT. You'd just have to set jit_provider = xyz, and it'd
work. If we made the generic JIT code depend on LLVM that'd end up
working pretty weirdly.  I guess we could set jit_provider = ''
something if instead of hardcoding llvmjit if LLVM is disabled.


> If you configured --without-llvm then the resulting core code is (one
> hopes) entirely incapable of using JIT, and it'd be better if the GUC
> settings reflected that.

That's not really the case, no. It controls whether the LLVM using jit
provider is built, not whether the generic JIT handling code is
available.  Given that the generic code has no dependencies, there seems
little reason to do that differently?

Greetings,

Andres Freund



Re: PostgreSQL 12, JIT defaults

От
Andres Freund
Дата:
Hi,

On 2018-10-08 11:43:42 -0400, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > On October 8, 2018 8:03:56 AM PDT, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >> A look in guc.c shows that jit defaults to "on" whether or not JIT is
> >> enabled at compile time. 
> >> This seems, at best, rather user-unfriendly.
> >> And it's not like our conventions for other compile-option-affected
> >> GUCs, eg the SSL ones.
> 
> > That was intentional, even though it perhaps should be better documented. That allows a distro to build and
distributepg without llvm enabled, but then have the JIT package with all the dependencies separately. The pg yum
packagesdo so.
 
> 
> I'm not following.  A distro wishing to do that would configure
> --with-llvm, not without, and then simply split up the build results
> so that the JIT stuff is in a separate subpackage.

Well, that'd then leave you with one more state (LLVM configured but not
installed, LLVM configured and installed, LLVM disabled and arguably
LLVM disabled but installed), but more importantly if you compile
without llvm enabled, you can still install a different extension that
would do JIT. You'd just have to set jit_provider = xyz, and it'd
work. If we made the generic JIT code depend on LLVM that'd end up
working pretty weirdly.  I guess we could set jit_provider = ''
something if instead of hardcoding llvmjit if LLVM is disabled.


> If you configured --without-llvm then the resulting core code is (one
> hopes) entirely incapable of using JIT, and it'd be better if the GUC
> settings reflected that.

That's not really the case, no. It controls whether the LLVM using jit
provider is built, not whether the generic JIT handling code is
available.  Given that the generic code has no dependencies, there seems
little reason to do that differently?

Greetings,

Andres Freund


Re: PostgreSQL 12, JIT defaults

От
Pavel Stehule
Дата:


po 8. 10. 2018 v 17:59 odesílatel Andres Freund <andres@anarazel.de> napsal:
Hi,

On 2018-10-08 11:43:42 -0400, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > On October 8, 2018 8:03:56 AM PDT, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >> A look in guc.c shows that jit defaults to "on" whether or not JIT is
> >> enabled at compile time.
> >> This seems, at best, rather user-unfriendly.
> >> And it's not like our conventions for other compile-option-affected
> >> GUCs, eg the SSL ones.
>
> > That was intentional, even though it perhaps should be better documented. That allows a distro to build and distribute pg without llvm enabled, but then have the JIT package with all the dependencies separately. The pg yum packages do so.
>
> I'm not following.  A distro wishing to do that would configure
> --with-llvm, not without, and then simply split up the build results
> so that the JIT stuff is in a separate subpackage.

Well, that'd then leave you with one more state (LLVM configured but not
installed, LLVM configured and installed, LLVM disabled and arguably
LLVM disabled but installed), but more importantly if you compile
without llvm enabled, you can still install a different extension that
would do JIT. You'd just have to set jit_provider = xyz, and it'd
work. If we made the generic JIT code depend on LLVM that'd end up
working pretty weirdly.  I guess we could set jit_provider = ''
something if instead of hardcoding llvmjit if LLVM is disabled.


> If you configured --without-llvm then the resulting core code is (one
> hopes) entirely incapable of using JIT, and it'd be better if the GUC
> settings reflected that..

That's not really the case, no. It controls whether the LLVM using jit
provider is built, not whether the generic JIT handling code is
available.  Given that the generic code has no dependencies, there seems
little reason to do that differently?

I can accept this logic, but it looks very fragile. Can be there some safeguard against using wrong version or wrong API?



Greetings,

Andres Freund

Re: PostgreSQL 12, JIT defaults

От
Pavel Stehule
Дата:


po 8. 10. 2018 v 17:59 odesílatel Andres Freund <andres@anarazel.de> napsal:
Hi,

On 2018-10-08 11:43:42 -0400, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > On October 8, 2018 8:03:56 AM PDT, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >> A look in guc.c shows that jit defaults to "on" whether or not JIT is
> >> enabled at compile time.
> >> This seems, at best, rather user-unfriendly.
> >> And it's not like our conventions for other compile-option-affected
> >> GUCs, eg the SSL ones.
>
> > That was intentional, even though it perhaps should be better documented. That allows a distro to build and distribute pg without llvm enabled, but then have the JIT package with all the dependencies separately. The pg yum packages do so.
>
> I'm not following.  A distro wishing to do that would configure
> --with-llvm, not without, and then simply split up the build results
> so that the JIT stuff is in a separate subpackage.

Well, that'd then leave you with one more state (LLVM configured but not
installed, LLVM configured and installed, LLVM disabled and arguably
LLVM disabled but installed), but more importantly if you compile
without llvm enabled, you can still install a different extension that
would do JIT. You'd just have to set jit_provider = xyz, and it'd
work. If we made the generic JIT code depend on LLVM that'd end up
working pretty weirdly.  I guess we could set jit_provider = ''
something if instead of hardcoding llvmjit if LLVM is disabled.


> If you configured --without-llvm then the resulting core code is (one
> hopes) entirely incapable of using JIT, and it'd be better if the GUC
> settings reflected that..

That's not really the case, no. It controls whether the LLVM using jit
provider is built, not whether the generic JIT handling code is
available.  Given that the generic code has no dependencies, there seems
little reason to do that differently?

I can accept this logic, but it looks very fragile. Can be there some safeguard against using wrong version or wrong API?



Greetings,

Andres Freund

Re: PostgreSQL 12, JIT defaults

От
Andres Freund
Дата:

On October 8, 2018 10:16:54 AM PDT, Pavel Stehule <pavel.stehule@gmail.com> wrote:
>po 8. 10. 2018 v 17:59 odesílatel Andres Freund <andres@anarazel.de>
>napsal:
>
>> Hi,
>>
>> On 2018-10-08 11:43:42 -0400, Tom Lane wrote:
>> > Andres Freund <andres@anarazel.de> writes:
>> > > On October 8, 2018 8:03:56 AM PDT, Tom Lane <tgl@sss.pgh.pa.us>
>wrote:
>> > >> A look in guc.c shows that jit defaults to "on" whether or not
>JIT is
>> > >> enabled at compile time.
>> > >> This seems, at best, rather user-unfriendly.
>> > >> And it's not like our conventions for other
>compile-option-affected
>> > >> GUCs, eg the SSL ones.
>> >
>> > > That was intentional, even though it perhaps should be better
>> documented. That allows a distro to build and distribute pg without
>llvm
>> enabled, but then have the JIT package with all the dependencies
>> separately. The pg yum packages do so.
>> >
>> > I'm not following.  A distro wishing to do that would configure
>> > --with-llvm, not without, and then simply split up the build
>results
>> > so that the JIT stuff is in a separate subpackage.
>>
>> Well, that'd then leave you with one more state (LLVM configured but
>not
>> installed, LLVM configured and installed, LLVM disabled and arguably
>> LLVM disabled but installed), but more importantly if you compile
>> without llvm enabled, you can still install a different extension
>that
>> would do JIT. You'd just have to set jit_provider = xyz, and it'd
>> work. If we made the generic JIT code depend on LLVM that'd end up
>> working pretty weirdly.  I guess we could set jit_provider = ''
>> something if instead of hardcoding llvmjit if LLVM is disabled.
>>
>
>>
>> > If you configured --without-llvm then the resulting core code is
>(one
>> > hopes) entirely incapable of using JIT, and it'd be better if the
>GUC
>> > settings reflected that..
>>
>> That's not really the case, no. It controls whether the LLVM using
>jit
>> provider is built, not whether the generic JIT handling code is
>> available.  Given that the generic code has no dependencies, there
>seems
>> little reason to do that differently?
>>
>
>I can accept this logic, but it looks very fragile. Can be there some
>safeguard against using wrong version or wrong API?

To me that seems like an llvm / JIT independent piece of infrastructure.  It'd probably be good if we had a catversion
likething to track ABI compatibility, but how to do so without making development unduly painful is less clear to me. 

Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.


Re: PostgreSQL 12, JIT defaults

От
Andres Freund
Дата:

On October 8, 2018 10:16:54 AM PDT, Pavel Stehule <pavel.stehule@gmail.com> wrote:
>po 8. 10. 2018 v 17:59 odesílatel Andres Freund <andres@anarazel.de>
>napsal:
>
>> Hi,
>>
>> On 2018-10-08 11:43:42 -0400, Tom Lane wrote:
>> > Andres Freund <andres@anarazel.de> writes:
>> > > On October 8, 2018 8:03:56 AM PDT, Tom Lane <tgl@sss.pgh.pa.us>
>wrote:
>> > >> A look in guc.c shows that jit defaults to "on" whether or not
>JIT is
>> > >> enabled at compile time.
>> > >> This seems, at best, rather user-unfriendly.
>> > >> And it's not like our conventions for other
>compile-option-affected
>> > >> GUCs, eg the SSL ones.
>> >
>> > > That was intentional, even though it perhaps should be better
>> documented. That allows a distro to build and distribute pg without
>llvm
>> enabled, but then have the JIT package with all the dependencies
>> separately. The pg yum packages do so.
>> >
>> > I'm not following.  A distro wishing to do that would configure
>> > --with-llvm, not without, and then simply split up the build
>results
>> > so that the JIT stuff is in a separate subpackage.
>>
>> Well, that'd then leave you with one more state (LLVM configured but
>not
>> installed, LLVM configured and installed, LLVM disabled and arguably
>> LLVM disabled but installed), but more importantly if you compile
>> without llvm enabled, you can still install a different extension
>that
>> would do JIT. You'd just have to set jit_provider = xyz, and it'd
>> work. If we made the generic JIT code depend on LLVM that'd end up
>> working pretty weirdly.  I guess we could set jit_provider = ''
>> something if instead of hardcoding llvmjit if LLVM is disabled.
>>
>
>>
>> > If you configured --without-llvm then the resulting core code is
>(one
>> > hopes) entirely incapable of using JIT, and it'd be better if the
>GUC
>> > settings reflected that..
>>
>> That's not really the case, no. It controls whether the LLVM using
>jit
>> provider is built, not whether the generic JIT handling code is
>> available.  Given that the generic code has no dependencies, there
>seems
>> little reason to do that differently?
>>
>
>I can accept this logic, but it looks very fragile. Can be there some
>safeguard against using wrong version or wrong API?

To me that seems like an llvm / JIT independent piece of infrastructure.  It'd probably be good if we had a catversion
likething to track ABI compatibility, but how to do so without making development unduly painful is less clear to me. 

Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.



Re: PostgreSQL 12, JIT defaults

От
Pavel Stehule
Дата:


po 8. 10. 2018 v 19:24 odesílatel Andres Freund <andres@anarazel.de> napsal:


On October 8, 2018 10:16:54 AM PDT, Pavel Stehule <pavel.stehule@gmail.com> wrote:
>po 8. 10. 2018 v 17:59 odesílatel Andres Freund <andres@anarazel.de>
>napsal:
>
>> Hi,
>>
>> On 2018-10-08 11:43:42 -0400, Tom Lane wrote:
>> > Andres Freund <andres@anarazel.de> writes:
>> > > On October 8, 2018 8:03:56 AM PDT, Tom Lane <tgl@sss.pgh.pa.us>
>wrote:
>> > >> A look in guc.c shows that jit defaults to "on" whether or not
>JIT is
>> > >> enabled at compile time.
>> > >> This seems, at best, rather user-unfriendly.
>> > >> And it's not like our conventions for other
>compile-option-affected
>> > >> GUCs, eg the SSL ones.
>> >
>> > > That was intentional, even though it perhaps should be better
>> documented. That allows a distro to build and distribute pg without
>llvm
>> enabled, but then have the JIT package with all the dependencies
>> separately. The pg yum packages do so.
>> >
>> > I'm not following.  A distro wishing to do that would configure
>> > --with-llvm, not without, and then simply split up the build
>results
>> > so that the JIT stuff is in a separate subpackage.
>>
>> Well, that'd then leave you with one more state (LLVM configured but
>not
>> installed, LLVM configured and installed, LLVM disabled and arguably
>> LLVM disabled but installed), but more importantly if you compile
>> without llvm enabled, you can still install a different extension
>that
>> would do JIT. You'd just have to set jit_provider = xyz, and it'd
>> work. If we made the generic JIT code depend on LLVM that'd end up
>> working pretty weirdly.  I guess we could set jit_provider = ''
>> something if instead of hardcoding llvmjit if LLVM is disabled.
>>
>
>>
>> > If you configured --without-llvm then the resulting core code is
>(one
>> > hopes) entirely incapable of using JIT, and it'd be better if the
>GUC
>> > settings reflected that..
>>
>> That's not really the case, no. It controls whether the LLVM using
>jit
>> provider is built, not whether the generic JIT handling code is
>> available.  Given that the generic code has no dependencies, there
>seems
>> little reason to do that differently?
>>
>
>I can accept this logic, but it looks very fragile. Can be there some
>safeguard against using wrong version or wrong API?

To me that seems like an llvm / JIT independent piece of infrastructure.  It'd probably be good if we had a catversion like thing to track ABI compatibility, but how to do so without making development unduly painful is less clear to me.

I am thinking so simple number should be good enough. We can require equality - Just I need a signal so some is wrong - better than Postgres crash.
 

Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: PostgreSQL 12, JIT defaults

От
Pavel Stehule
Дата:


po 8. 10. 2018 v 19:24 odesílatel Andres Freund <andres@anarazel.de> napsal:


On October 8, 2018 10:16:54 AM PDT, Pavel Stehule <pavel.stehule@gmail.com> wrote:
>po 8. 10. 2018 v 17:59 odesílatel Andres Freund <andres@anarazel.de>
>napsal:
>
>> Hi,
>>
>> On 2018-10-08 11:43:42 -0400, Tom Lane wrote:
>> > Andres Freund <andres@anarazel.de> writes:
>> > > On October 8, 2018 8:03:56 AM PDT, Tom Lane <tgl@sss.pgh.pa.us>
>wrote:
>> > >> A look in guc.c shows that jit defaults to "on" whether or not
>JIT is
>> > >> enabled at compile time.
>> > >> This seems, at best, rather user-unfriendly.
>> > >> And it's not like our conventions for other
>compile-option-affected
>> > >> GUCs, eg the SSL ones.
>> >
>> > > That was intentional, even though it perhaps should be better
>> documented. That allows a distro to build and distribute pg without
>llvm
>> enabled, but then have the JIT package with all the dependencies
>> separately. The pg yum packages do so.
>> >
>> > I'm not following.  A distro wishing to do that would configure
>> > --with-llvm, not without, and then simply split up the build
>results
>> > so that the JIT stuff is in a separate subpackage.
>>
>> Well, that'd then leave you with one more state (LLVM configured but
>not
>> installed, LLVM configured and installed, LLVM disabled and arguably
>> LLVM disabled but installed), but more importantly if you compile
>> without llvm enabled, you can still install a different extension
>that
>> would do JIT. You'd just have to set jit_provider = xyz, and it'd
>> work. If we made the generic JIT code depend on LLVM that'd end up
>> working pretty weirdly.  I guess we could set jit_provider = ''
>> something if instead of hardcoding llvmjit if LLVM is disabled.
>>
>
>>
>> > If you configured --without-llvm then the resulting core code is
>(one
>> > hopes) entirely incapable of using JIT, and it'd be better if the
>GUC
>> > settings reflected that..
>>
>> That's not really the case, no. It controls whether the LLVM using
>jit
>> provider is built, not whether the generic JIT handling code is
>> available.  Given that the generic code has no dependencies, there
>seems
>> little reason to do that differently?
>>
>
>I can accept this logic, but it looks very fragile. Can be there some
>safeguard against using wrong version or wrong API?

To me that seems like an llvm / JIT independent piece of infrastructure.  It'd probably be good if we had a catversion like thing to track ABI compatibility, but how to do so without making development unduly painful is less clear to me.

I am thinking so simple number should be good enough. We can require equality - Just I need a signal so some is wrong - better than Postgres crash.
 

Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: PostgreSQL 12, JIT defaults

От
Andres Freund
Дата:

On October 8, 2018 10:29:56 AM PDT, Pavel Stehule <pavel.stehule@gmail.com> wrote:
>po 8. 10. 2018 v 19:24 odesílatel Andres Freund <andres@anarazel.de>
>napsal:
>
>>
>>
>> On October 8, 2018 10:16:54 AM PDT, Pavel Stehule
><pavel.stehule@gmail.com>
>> wrote:
>> >po 8. 10. 2018 v 17:59 odesílatel Andres Freund <andres@anarazel.de>
>> >napsal:
>> >
>> >> Hi,
>> >>
>> >> On 2018-10-08 11:43:42 -0400, Tom Lane wrote:
>> >> > Andres Freund <andres@anarazel.de> writes:
>> >> > > On October 8, 2018 8:03:56 AM PDT, Tom Lane
><tgl@sss.pgh.pa.us>
>> >wrote:
>> >> > >> A look in guc.c shows that jit defaults to "on" whether or
>not
>> >JIT is
>> >> > >> enabled at compile time.
>> >> > >> This seems, at best, rather user-unfriendly.
>> >> > >> And it's not like our conventions for other
>> >compile-option-affected
>> >> > >> GUCs, eg the SSL ones.
>> >> >
>> >> > > That was intentional, even though it perhaps should be better
>> >> documented. That allows a distro to build and distribute pg
>without
>> >llvm
>> >> enabled, but then have the JIT package with all the dependencies
>> >> separately. The pg yum packages do so.
>> >> >
>> >> > I'm not following.  A distro wishing to do that would configure
>> >> > --with-llvm, not without, and then simply split up the build
>> >results
>> >> > so that the JIT stuff is in a separate subpackage.
>> >>
>> >> Well, that'd then leave you with one more state (LLVM configured
>but
>> >not
>> >> installed, LLVM configured and installed, LLVM disabled and
>arguably
>> >> LLVM disabled but installed), but more importantly if you compile
>> >> without llvm enabled, you can still install a different extension
>> >that
>> >> would do JIT. You'd just have to set jit_provider = xyz, and it'd
>> >> work. If we made the generic JIT code depend on LLVM that'd end up
>> >> working pretty weirdly.  I guess we could set jit_provider = ''
>> >> something if instead of hardcoding llvmjit if LLVM is disabled.
>> >>
>> >
>> >>
>> >> > If you configured --without-llvm then the resulting core code is
>> >(one
>> >> > hopes) entirely incapable of using JIT, and it'd be better if
>the
>> >GUC
>> >> > settings reflected that..
>> >>
>> >> That's not really the case, no. It controls whether the LLVM using
>> >jit
>> >> provider is built, not whether the generic JIT handling code is
>> >> available.  Given that the generic code has no dependencies, there
>> >seems
>> >> little reason to do that differently?
>> >>
>> >
>> >I can accept this logic, but it looks very fragile. Can be there
>some
>> >safeguard against using wrong version or wrong API?
>>
>> To me that seems like an llvm / JIT independent piece of
>infrastructure.
>> It'd probably be good if we had a catversion like thing to track ABI
>> compatibility, but how to do so without making development unduly
>painful
>> is less clear to me.
>>
>
>I am thinking so simple number should be good enough. We can require
>equality - Just I need a signal so some is wrong - better than Postgres
>crash.

It'd cause constant conflicts and / or we would regularly forget updating it. It's not that trivial to determine what
influencesABI compatibility. 

Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.


Re: PostgreSQL 12, JIT defaults

От
Andres Freund
Дата:

On October 8, 2018 10:29:56 AM PDT, Pavel Stehule <pavel.stehule@gmail.com> wrote:
>po 8. 10. 2018 v 19:24 odesílatel Andres Freund <andres@anarazel.de>
>napsal:
>
>>
>>
>> On October 8, 2018 10:16:54 AM PDT, Pavel Stehule
><pavel.stehule@gmail.com>
>> wrote:
>> >po 8. 10. 2018 v 17:59 odesílatel Andres Freund <andres@anarazel.de>
>> >napsal:
>> >
>> >> Hi,
>> >>
>> >> On 2018-10-08 11:43:42 -0400, Tom Lane wrote:
>> >> > Andres Freund <andres@anarazel.de> writes:
>> >> > > On October 8, 2018 8:03:56 AM PDT, Tom Lane
><tgl@sss.pgh.pa.us>
>> >wrote:
>> >> > >> A look in guc.c shows that jit defaults to "on" whether or
>not
>> >JIT is
>> >> > >> enabled at compile time.
>> >> > >> This seems, at best, rather user-unfriendly.
>> >> > >> And it's not like our conventions for other
>> >compile-option-affected
>> >> > >> GUCs, eg the SSL ones.
>> >> >
>> >> > > That was intentional, even though it perhaps should be better
>> >> documented. That allows a distro to build and distribute pg
>without
>> >llvm
>> >> enabled, but then have the JIT package with all the dependencies
>> >> separately. The pg yum packages do so.
>> >> >
>> >> > I'm not following.  A distro wishing to do that would configure
>> >> > --with-llvm, not without, and then simply split up the build
>> >results
>> >> > so that the JIT stuff is in a separate subpackage.
>> >>
>> >> Well, that'd then leave you with one more state (LLVM configured
>but
>> >not
>> >> installed, LLVM configured and installed, LLVM disabled and
>arguably
>> >> LLVM disabled but installed), but more importantly if you compile
>> >> without llvm enabled, you can still install a different extension
>> >that
>> >> would do JIT. You'd just have to set jit_provider = xyz, and it'd
>> >> work. If we made the generic JIT code depend on LLVM that'd end up
>> >> working pretty weirdly.  I guess we could set jit_provider = ''
>> >> something if instead of hardcoding llvmjit if LLVM is disabled.
>> >>
>> >
>> >>
>> >> > If you configured --without-llvm then the resulting core code is
>> >(one
>> >> > hopes) entirely incapable of using JIT, and it'd be better if
>the
>> >GUC
>> >> > settings reflected that..
>> >>
>> >> That's not really the case, no. It controls whether the LLVM using
>> >jit
>> >> provider is built, not whether the generic JIT handling code is
>> >> available.  Given that the generic code has no dependencies, there
>> >seems
>> >> little reason to do that differently?
>> >>
>> >
>> >I can accept this logic, but it looks very fragile. Can be there
>some
>> >safeguard against using wrong version or wrong API?
>>
>> To me that seems like an llvm / JIT independent piece of
>infrastructure.
>> It'd probably be good if we had a catversion like thing to track ABI
>> compatibility, but how to do so without making development unduly
>painful
>> is less clear to me.
>>
>
>I am thinking so simple number should be good enough. We can require
>equality - Just I need a signal so some is wrong - better than Postgres
>crash.

It'd cause constant conflicts and / or we would regularly forget updating it. It's not that trivial to determine what
influencesABI compatibility. 

Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.



Re: PostgreSQL 12, JIT defaults

От
Tom Lane
Дата:
Andres Freund <andres@anarazel.de> writes:
>> I am thinking so simple number should be good enough. We can require
>> equality - Just I need a signal so some is wrong - better than Postgres
>> crash.

> It'd cause constant conflicts and / or we would regularly forget updating it. It's not that trivial to determine what
influencesABI compatibility. 

There already is a PG major-version-number check embedded in the
PG_MODULE_MAGIC infrastructure, which is plenty for regular users.
It's not sufficient for developers working with HEAD, of course.

We could consider making that work off of catversion instead, but I don't
think it'd really improve matters to do so.  catversion doesn't cover most
of what can break loadable modules, such as changes of Node data
structures.

            regards, tom lane



Re: PostgreSQL 12, JIT defaults

От
Tom Lane
Дата:
Andres Freund <andres@anarazel.de> writes:
>> I am thinking so simple number should be good enough. We can require
>> equality - Just I need a signal so some is wrong - better than Postgres
>> crash.

> It'd cause constant conflicts and / or we would regularly forget updating it. It's not that trivial to determine what
influencesABI compatibility. 

There already is a PG major-version-number check embedded in the
PG_MODULE_MAGIC infrastructure, which is plenty for regular users.
It's not sufficient for developers working with HEAD, of course.

We could consider making that work off of catversion instead, but I don't
think it'd really improve matters to do so.  catversion doesn't cover most
of what can break loadable modules, such as changes of Node data
structures.

            regards, tom lane


Re: PostgreSQL 12, JIT defaults

От
Pavel Stehule
Дата:



>
>I am thinking so simple number should be good enough. We can require
>equality - Just I need a signal so some is wrong - better than Postgres
>crash.

It'd cause constant conflicts and / or we would regularly forget updating it. It's not that trivial to determine what influences ABI compatibility.

This can be checked by regress tests? I don't know. Maybe I am not too friendly, I am sorry. I spent 20 minutes by searching phantom, because JIT was active, although I wanted.

So any help against this situation can be welcome.

Regards

Pavel


Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: PostgreSQL 12, JIT defaults

От
Pavel Stehule
Дата:



>
>I am thinking so simple number should be good enough. We can require
>equality - Just I need a signal so some is wrong - better than Postgres
>crash.

It'd cause constant conflicts and / or we would regularly forget updating it. It's not that trivial to determine what influences ABI compatibility.

This can be checked by regress tests? I don't know. Maybe I am not too friendly, I am sorry. I spent 20 minutes by searching phantom, because JIT was active, although I wanted.

So any help against this situation can be welcome.

Regards

Pavel


Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.