Обсуждение: improving PL/Python builds on OS X
The PL/Python build on OS X is currently hardcoded to use the system Python install. If you try to override this when running configure, you get a mysterious mix-and-match build. If you want to build against your own Python build, or MacPorts or Homebrew, you can't. This is straightforward to fix. In configure, besides checking Python include and library paths, we can also check whether it's a framework build and the right parameters for that. The attached patch does that and does the job for me. Please test it. One constraint, which is explained in the comment in src/pl/plpython/Makefile is that in Python <2.5, there is no official way to detect either framework builds or shared libpython builds, so we can't support those versions on OS X, at least without more hardcoding of things. I'd rather phase some of that out, but if someone needs to continue to use Python 2.4 or earlier on OS X, let me know. (Or more proper fix would be to split DLSUFFIX into two variables, but that seems more work than it's worth right now.)
Вложения
On Fri, Dec 21, 2012 at 5:45 PM, Peter Eisentraut <peter_e@gmx.net> wrote: > The PL/Python build on OS X is currently hardcoded to use the system > Python install. If you try to override this when running configure, you > get a mysterious mix-and-match build. If you want to build against your > own Python build, or MacPorts or Homebrew, you can't. > > This is straightforward to fix. In configure, besides checking Python > include and library paths, we can also check whether it's a framework > build and the right parameters for that. The attached patch does that > and does the job for me. Please test it. > > One constraint, which is explained in the comment in > src/pl/plpython/Makefile is that in Python <2.5, there is no official > way to detect either framework builds or shared libpython builds, so we > can't support those versions on OS X, at least without more hardcoding > of things. I'd rather phase some of that out, but if someone needs to > continue to use Python 2.4 or earlier on OS X, let me know. (Or more > proper fix would be to split DLSUFFIX into two variables, but that seems > more work than it's worth right now.) This patch seems to have broken the build for our installers for 9.3. Because we need a consistent build of the PL interpretors on all the platforms we support, we use the ActiveState distributions of Perl, Python and TCL (we can't rely on vendor supplied packages, because their versions vary between different Linux distros and different OS X versions). However, ActivePython doesn't include a shared library, which this change seems to require. Can that requirement be reverted? -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On 5/8/13 11:51 AM, Dave Page wrote: > This patch seems to have broken the build for our installers for 9.3. > Because we need a consistent build of the PL interpretors on all the > platforms we support, we use the ActiveState distributions of Perl, > Python and TCL (we can't rely on vendor supplied packages, because > their versions vary between different Linux distros and different OS X > versions). However, ActivePython doesn't include a shared library, > which this change seems to require. > > Can that requirement be reverted? There was no change in this regard. A shared library was always required on OS X. ActivePython does include a shared library. I just tried it and it builds fine.
On Wed, May 8, 2013 at 5:34 PM, Peter Eisentraut <peter_e@gmx.net> wrote: > On 5/8/13 11:51 AM, Dave Page wrote: >> This patch seems to have broken the build for our installers for 9.3. >> Because we need a consistent build of the PL interpretors on all the >> platforms we support, we use the ActiveState distributions of Perl, >> Python and TCL (we can't rely on vendor supplied packages, because >> their versions vary between different Linux distros and different OS X >> versions). However, ActivePython doesn't include a shared library, >> which this change seems to require. >> >> Can that requirement be reverted? > > There was no change in this regard. A shared library was always > required on OS X. It's failing on Linux. Even worse, it configures fine and then builds without error. There is a message spewed out by configure, but it doesn't contain the words warning or error. Given that I explicitly said I wanted Python support when I ran configure, it should certainly fail with an error at configure time. We only noticed this was a problem when the QA guys started diving in to more detailed tested, as we don't watch for every message in the 50+ MB of logs our automated build systems generate. > ActivePython does include a shared library. I just tried it and it > builds fine. -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On Wed, 2013-05-08 at 18:24 +0100, Dave Page wrote: > It's failing on Linux. Even worse, it configures fine and then builds > without error. There is a message spewed out by configure, but it > doesn't contain the words warning or error. Given that I explicitly > said I wanted Python support when I ran configure, it should certainly > fail with an error at configure time. We only noticed this was a > problem when the QA guys started diving in to more detailed tested, as > we don't watch for every message in the 50+ MB of logs our automated > build systems generate. It worked before because we used to allow linking shared libraries against static libraries on some platforms. But that was more or less a lie because it doesn't work on 64-bit platforms. ActiveState Python contains a static library with PIC files. There is no obvious way to detect that, which is why we don't support it directly. You can sort it out yourself by building with make shared_libpython=yes In the long term, find a way to detect whether the library is usable.
On Thu, May 9, 2013 at 2:26 AM, Peter Eisentraut <peter_e@gmx.net> wrote: > On Wed, 2013-05-08 at 18:24 +0100, Dave Page wrote: >> It's failing on Linux. Even worse, it configures fine and then builds >> without error. There is a message spewed out by configure, but it >> doesn't contain the words warning or error. Given that I explicitly >> said I wanted Python support when I ran configure, it should certainly >> fail with an error at configure time. We only noticed this was a >> problem when the QA guys started diving in to more detailed tested, as >> we don't watch for every message in the 50+ MB of logs our automated >> build systems generate. > > It worked before because we used to allow linking shared libraries > against static libraries on some platforms. But that was more or less a > lie because it doesn't work on 64-bit platforms. > > ActiveState Python contains a static library with PIC files. There is > no obvious way to detect that, which is why we don't support it > directly. You can sort it out yourself by building with > > make shared_libpython=yes OK, thanks - I'll try that. I assume you'll fix the configure script so it actually errors out if plpython cannot be built, but is explicitly requested? -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On Thu, May 9, 2013 at 8:09 AM, Dave Page <dpage@pgadmin.org> wrote: > On Thu, May 9, 2013 at 2:26 AM, Peter Eisentraut <peter_e@gmx.net> wrote: >> On Wed, 2013-05-08 at 18:24 +0100, Dave Page wrote: >>> It's failing on Linux. Even worse, it configures fine and then builds >>> without error. There is a message spewed out by configure, but it >>> doesn't contain the words warning or error. Given that I explicitly >>> said I wanted Python support when I ran configure, it should certainly >>> fail with an error at configure time. We only noticed this was a >>> problem when the QA guys started diving in to more detailed tested, as >>> we don't watch for every message in the 50+ MB of logs our automated >>> build systems generate. >> >> It worked before because we used to allow linking shared libraries >> against static libraries on some platforms. But that was more or less a >> lie because it doesn't work on 64-bit platforms. >> >> ActiveState Python contains a static library with PIC files. There is >> no obvious way to detect that, which is why we don't support it >> directly. You can sort it out yourself by building with >> >> make shared_libpython=yes > > OK, thanks - I'll try that. > > I assume you'll fix the configure script so it actually errors out if > plpython cannot be built, but is explicitly requested? BTW - it's always worked fine for us on 64 bit machines with the past few major releases of both PG and Python - are you saying that's pure chance? For example: postgres=# CREATE LANGUAGE plpython3u; CREATE LANGUAGE postgres=# CREATE FUNCTION pyversion() RETURNS text AS postgres-# $$ postgres$# import sys postgres$# return sys.version postgres$# $$ LANGUAGE 'plpython3u'; CREATE FUNCTION postgres=# SELECT pyversion(); pyversion -----------------------------------------3.2.2 (default, Sep 8 2011, 12:20:18) +[GCC 4.0.2 20051125 (Red Hat 4.0.2-8)] (1 row) postgres=# SELECT version(); version ---------------------------------------------------------------------------------------------------------------PostgreSQL 9.2.4on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-52), 64-bit (1 row) postgres=# (that's with fresh installations of ActivePython 3.2.2.3 and EDB's PostgreSQL 9.2.4 package, on CentOS 6). -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On 5/9/13 3:09 AM, Dave Page wrote: > I assume you'll fix the configure script so it actually errors out if > plpython cannot be built, but is explicitly requested? That is ancient behavior, which I'm not fond of, but now is not the time to change that.
On 5/9/13 3:25 AM, Dave Page wrote: > BTW - it's always worked fine for us on 64 bit machines with the past > few major releases of both PG and Python - are you saying that's pure > chance? It works because ActiveState Python has PIC inside a static library. But we have no straightforward way of knowing that (AFAIK), other than observing whether the build result crashes or not.
On Thu, May 9, 2013 at 3:09 PM, Peter Eisentraut <peter_e@gmx.net> wrote: > On 5/9/13 3:09 AM, Dave Page wrote: >> I assume you'll fix the configure script so it actually errors out if >> plpython cannot be built, but is explicitly requested? > > That is ancient behavior, which I'm not fond of, but now is not the time > to change that. Eh? Beta is precisely the time to find and fix bugs like this. -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On 5/9/13 10:52 AM, Dave Page wrote: > On Thu, May 9, 2013 at 3:09 PM, Peter Eisentraut <peter_e@gmx.net> wrote: >> On 5/9/13 3:09 AM, Dave Page wrote: >>> I assume you'll fix the configure script so it actually errors out if >>> plpython cannot be built, but is explicitly requested? >> >> That is ancient behavior, which I'm not fond of, but now is not the time >> to change that. > > Eh? Beta is precisely the time to find and fix bugs like this. It's not a bug, it's working as designed (not by me). It's certainly worth reexamining, but there are so many variations in this area (and Perl and Tcl are set up the same way) that I don't want to mess around in it now. A better approach for now would be to look for a configuration item in Python that tells us whether libpython contains PIC.
On Thu, May 9, 2013 at 10:06 PM, Peter Eisentraut <peter_e@gmx.net> wrote: > On 5/9/13 10:52 AM, Dave Page wrote: >> On Thu, May 9, 2013 at 3:09 PM, Peter Eisentraut <peter_e@gmx.net> wrote: >>> On 5/9/13 3:09 AM, Dave Page wrote: >>>> I assume you'll fix the configure script so it actually errors out if >>>> plpython cannot be built, but is explicitly requested? >>> >>> That is ancient behavior, which I'm not fond of, but now is not the time >>> to change that. >> >> Eh? Beta is precisely the time to find and fix bugs like this. > > It's not a bug, it's working as designed (not by me). Well, design bug then. It's clearly a nasty POLA violation at minimum. -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On Thu, 2013-05-09 at 10:11 -0400, Peter Eisentraut wrote: > On 5/9/13 3:25 AM, Dave Page wrote: > > BTW - it's always worked fine for us on 64 bit machines with the past > > few major releases of both PG and Python - are you saying that's pure > > chance? > > It works because ActiveState Python has PIC inside a static library. > But we have no straightforward way of knowing that (AFAIK), other than > observing whether the build result crashes or not. After further digging, it seems to me that their build is not a standard build. They must be patching in compiler options through the backdoor somehow. Their config/Makefile has BASECFLAGS= -fno-strict-aliasing -fPIC meaning that they compile *everything* with those options. But that's not something that the standard Python configure script can produce. Again, reliably detecting that might be difficult.
On Fri, May 10, 2013 at 2:47 AM, Peter Eisentraut <peter_e@gmx.net> wrote: > On Thu, 2013-05-09 at 10:11 -0400, Peter Eisentraut wrote: >> On 5/9/13 3:25 AM, Dave Page wrote: >> > BTW - it's always worked fine for us on 64 bit machines with the past >> > few major releases of both PG and Python - are you saying that's pure >> > chance? >> >> It works because ActiveState Python has PIC inside a static library. >> But we have no straightforward way of knowing that (AFAIK), other than >> observing whether the build result crashes or not. > > After further digging, it seems to me that their build is not a standard > build. They must be patching in compiler options through the backdoor > somehow. Their config/Makefile has > > BASECFLAGS= -fno-strict-aliasing -fPIC > > meaning that they compile *everything* with those options. But that's > not something that the standard Python configure script can produce. Huh, interesting. I used to have a contact at ActiveState - I'll see if I can get some more details. > Again, reliably detecting that might be difficult. Yeah, that's understandable. As long as I can force it though (and your suggestion on that front seems to work fine), I'm happy. Thanks. -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company