Обсуждение: clang prefix for centos7
I'm migating a postgres11 server to centos7. It turns out we need parray_gin extension. Under centos6, it compiled and installed fine, but but under c7 it fails: sudo pgxnclient install parray_gin |INFO: best version: parray_gin 1.3.1 |INFO: saving /tmp/tmpyrOi8w/parray_gin-1.3.1.zip |INFO: unpacking: /tmp/tmpyrOi8w/parray_gin-1.3.1.zip |INFO: building extension |gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security-fno-strict-aliasing -fwrapv -fexcess-precision=standard -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions-fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC -I. -I./ -I/usr/pgsql-11/include/server-I/usr/pgsql-11/include/internal -D_GNU_SOURCE -I/usr/include/libxml2 -I/usr/include -c-o src/parray_gin.o src/parray_gin.c |gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security-fno-strict-aliasing -fwrapv -fexcess-precision=standard -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions-fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC -I. -I./ -I/usr/pgsql-11/include/server-I/usr/pgsql-11/include/internal -D_GNU_SOURCE -I/usr/include/libxml2 -I/usr/include -c-o src/trgm.o src/trgm.c |gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security-fno-strict-aliasing -fwrapv -fexcess-precision=standard -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions-fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC -shared -oparray_gin.so src/parray_gin.o src/trgm.o -L/usr/pgsql-11/lib -Wl,--as-needed -L/usr/lib64/llvm5.0/lib -L/usr/lib64 -Wl,--as-needed-Wl,-rpath,'/usr/pgsql-11/lib',--enable-new-dtags |cp sql/parray_gin.sql sql/parray_gin--1.3.1.sql |/opt/rh/llvm-toolset-7/root/usr/bin/clang -Wno-ignored-attributes -fno-strict-aliasing -fwrapv -O2 -I. -I./ -I/usr/pgsql-11/include/server-I/usr/pgsql-11/include/internal -D_GNU_SOURCE -I/usr/include/libxml2 -I/usr/include -flto=thin-emit-llvm -c -o src/parray_gin.bc src/parray_gin.c |gmake: /opt/rh/llvm-toolset-7/root/usr/bin/clang: Command not found |gmake: *** [src/parray_gin.bc] Error 127 |ERROR: command returned 2: gmake PG_CONFIG=/bin/pg_config all I guess that's because of: [pryzbyj@database parray_gin-1.3.1]$ /usr/pgsql-11/bin/pg_config CONFIGURE = ... 'CLANG=/opt/rh/llvm-toolset-7/root/usr/bin/clang' ... Searching for "/opt/rh/llvm-toolset-7/root/usr/bin/clang: Command not found", I see it's affected a bunch of people and a bunch of different extensions. Is it possible to compile with CLANG=/usr/bin/clang, instead ? Thanks, Justin
Justin Pryzby wrote: > I'm migating a postgres11 server to centos7. > It turns out we need parray_gin extension. > > Under centos6, it compiled and installed fine, but but under c7 it fails: > > sudo pgxnclient install parray_gin > > gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute-Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2-fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic-fPIC -I. -I./ -I/usr/pgsql-11/include/server -I/usr/pgsql-11/include/internal -D_GNU_SOURCE -I/usr/include/libxml2 -I/usr/include -c -o src/parray_gin.o src/parray_gin.c > > gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute-Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2-fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic-fPIC -I. -I./ -I/usr/pgsql-11/include/server -I/usr/pgsql-11/include/internal -D_GNU_SOURCE -I/usr/include/libxml2 -I/usr/include -c -o src/trgm.o src/trgm.c > > gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute-Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2-fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic-fPIC -shared -o parray_gin.so src/parray_gin.o src/trgm.o -L/usr/pgsql-11/lib -Wl,--as-needed -L/usr/lib64/llvm5.0/lib -L/usr/lib64 -Wl,--as-needed -Wl,-rpath,'/usr/pgsql-11/lib',--enable-new-dtags > > cp sql/parray_gin.sql sql/parray_gin--1.3.1.sql > > /opt/rh/llvm-toolset-7/root/usr/bin/clang -Wno-ignored-attributes -fno-strict-aliasing -fwrapv -O2 -I. -I./ -I/usr/pgsql-11/include/server-I/usr/pgsql-11/include/internal -D_GNU_SOURCE -I/usr/include/libxml2 -I/usr/include -flto=thin-emit-llvm -c -o src/parray_gin.bc src/parray_gin.c > > gmake: /opt/rh/llvm-toolset-7/root/usr/bin/clang: Command not found > > gmake: *** [src/parray_gin.bc] Error 127 > > ERROR: command returned 2: gmake PG_CONFIG=/bin/pg_config all If you don't need JIT support for the extension, you could try with_llvm=no make -e That worked with oracle_fdw. Don't know how that translates to pgxnclient though. Yours, Laurenz Albe -- Cybertec | https://www.cybertec-postgresql.com
Hi, On Sat, 2019-03-16 at 12:26 -0500, Justin Pryzby wrote: > Is it possible to compile with CLANG=/usr/bin/clang, instead ? No: # rpm -qf /opt/rh/llvm-toolset-7/root/usr/bin/clang-5.0 llvm-toolset-7-clang-5.0.1-4.el7.x86_64 # rpm -qv clang clang-3.4.2-9.el7.x86_64 The OS-default clang version is not sufficient enough to build PostgreSQL. Regards, -- Devrim Gündüz Open Source Solution Architect, Red Hat Certified Engineer Twitter: @DevrimGunduz , @DevrimGunduzTR
Вложения
> The OS-default clang version is not sufficient enough to build PostgreSQL. This is a packaging bug in the PGDG yum CentOS 7 packages. I reported it earlier; see https://www.postgresql.org/message-id/CAMsr+YGzuPv3qSBp3LCrc9SnYi=iHfijdjERNNufh75+YM-92g@mail.gmail.com . The centos7 postgresql11-devel package is broken. I'll reply with a followup mail with details so I don't upset the automailmod but in short installing the devel package gets you an unusable build env due to missing dependencies. It's seemingly broken for all Pg versions really due to not declaring a dependency on clang. It's just worse for centos 7 because you can't simply install the required clang. I originally tested adding explicit dependences on SCL's llvm toolset and clang, but that proved painful for other reasons. Especially since the steps to enable the SCL llvm toolset repo differ for CentOS 7 vs RHEL 7. So instead I propose that: * For CentOS 7, patch pgxs.mk to detect a missing clang and not add the bytecode dependency rules; * For OSes > CentOS 7 where we don't need to use SCL to get a suitable llvm, add a Requires: for the appropriate clang to the devel package, matching the BuildRequires specified by the source package. A patch for pgxs.mk is attached. I suggest adding it as a rpmbuild patch when building for centos 7. IMO the whole mess really a PGXS / build system issue where we bake the tool and compiler paths into our Makefile.global and pgxs.mk. But that's not simple to fix, since we have no guarantee that other tools found elsewhere will be compatible and produce correct code. I've seen a similar problem appear when the VM/container that built the postgres packages themselves used ccache, but ccache was not present on the machine that then installed the -devel package. At least that's easy to work around and defend against though. -- Craig Ringer http://www.2ndQuadrant.com/ 2ndQuadrant - PostgreSQL Solutions for the Enterprise
Вложения
On Fri, 6 Mar 2020 at 12:52, Craig Ringer <craig@2ndquadrant.com> wrote: > > > The OS-default clang version is not sufficient enough to build PostgreSQL. > > This is a packaging bug in the PGDG yum CentOS 7 packages. I reported > it earlier; see > https://www.postgresql.org/message-id/CAMsr+YGzuPv3qSBp3LCrc9SnYi=iHfijdjERNNufh75+YM-92g@mail.gmail.com > . > > The centos7 postgresql11-devel package is broken. I'll reply with a > followup mail with details so I don't upset the automailmod Repro/demo attached. Usage: $ mkdir centos7-llvm-report $ cd centos7-llvm-report $ # copy the attached files into . $ docker build -t centos7-llvm-report . $ docker run -it --name centos7-llvm-report centos7-llvm-report Result: [craig@linux pgdgrepor]$ docker run --name foo -it --rm foo This build will fail when trying to compile llvm bitcode gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC -I. -I./ -I/usr/pgsql-11/include/server -I/usr/pgsql-11/include/internal -D_GNU_SOURCE -I/usr/include/libxml2 -I/usr/include -c -o dummy.o dummy.c gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC -L/usr/pgsql-11/lib -Wl,--as-needed -L/usr/lib64/llvm5.0/lib -L/usr/lib64 -Wl,--as-needed -Wl,-rpath,'/usr/pgsql-11/lib',--enable-new-dtags -shared -o dummy.so dummy.o /opt/rh/llvm-toolset-7/root/usr/bin/clang -Wno-ignored-attributes -fno-strict-aliasing -fwrapv -O2 -I. -I./ -I/usr/pgsql-11/include/server -I/usr/pgsql-11/include/internal -D_GNU_SOURCE -I/usr/include/libxml2 -I/usr/include -flto=thin -emit-llvm -c -o dummy.bc dummy.c make: /opt/rh/llvm-toolset-7/root/usr/bin/clang: Command not found make: *** [dummy.bc] Error 127 This build will succeed gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC -I. -I./ -I/usr/pgsql-11/include/server -I/usr/pgsql-11/include/internal -D_GNU_SOURCE -I/usr/include/libxml2 -I/usr/include -c -o dummy.o dummy.c gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC -L/usr/pgsql-11/lib -Wl,--as-needed -L/usr/lib64/llvm5.0/lib -L/usr/lib64 -Wl,--as-needed -Wl,-rpath,'/usr/pgsql-11/lib',--enable-new-dtags -shared -o dummy.so dummy.o /usr/bin/mkdir -p '/usr/pgsql-11/lib' /usr/bin/install -c -m 755 dummy.so '/usr/pgsql-11/lib/'
Вложения
Hi Craig, Thanks for reporting this (again). Next time, please try report using redmine. That makes my life easier. For this case, I'm tracking it. Will commit changes tomorrow or so. Do you think this requires an update before next minor releases? Regards, Devrim On Fri, 2020-03-06 at 12:52 +0800, Craig Ringer wrote: > > The OS-default clang version is not sufficient enough to build PostgreSQL. > > This is a packaging bug in the PGDG yum CentOS 7 packages. I reported > it earlier; see > https://www.postgresql.org/message-id/CAMsr+YGzuPv3qSBp3LCrc9SnYi=iHfijdjERNNufh75+YM-92g@mail.gmail.com > . > > The centos7 postgresql11-devel package is broken. I'll reply with a > followup mail with details so I don't upset the automailmod > but in short installing the devel package gets you an unusable build > env due to missing dependencies. > > It's seemingly broken for all Pg versions really due to not declaring > a dependency on clang. It's just worse for centos 7 because you can't > simply install the required clang. I originally tested adding explicit > dependences on SCL's llvm toolset and clang, but that proved painful > for other reasons. Especially since the steps to enable the SCL llvm > toolset repo differ for CentOS 7 vs RHEL 7. > > So instead I propose that: > > * For CentOS 7, patch pgxs.mk to detect a missing clang and not add > the bytecode dependency rules; > * For OSes > CentOS 7 where we don't need to use SCL to get a suitable > llvm, add a Requires: for the appropriate clang to the devel package, > matching the BuildRequires specified by the source package. > > A patch for pgxs.mk is attached. I suggest adding it as a rpmbuild > patch when building for centos 7. > > IMO the whole mess really a PGXS / build system issue where we bake > the tool and compiler paths into our Makefile.global and pgxs.mk. But > that's not simple to fix, since we have no guarantee that other tools > found elsewhere will be compatible and produce correct code. I've seen > a similar problem appear when the VM/container that built the postgres > packages themselves used ccache, but ccache was not present on the > machine that then installed the -devel package. At least that's easy > to work around and defend against though. > > -- > Craig Ringer http://www.2ndQuadrant.com/ > 2ndQuadrant - PostgreSQL Solutions for the Enterprise -- Devrim Gündüz Open Source Solution Architect, Red Hat Certified Engineer Twitter: @DevrimGunduz , @DevrimGunduzTR
Вложения
On Wed, 11 Mar 2020, 09:33 Devrim Gündüz, <devrim@gunduz.org> wrote:
Hi Craig,
Thanks for reporting this (again). Next time, please try report using redmine.
That makes my life easier.
Can do.
Do you think this requires an update before next minor releases?
I'm surprised nobody's shouting about it, but they aren't. So it'll be fine for the next minor I think.
I suggest the Makefile.global workaround and not adding llvm dependencies.
I looked at adding weak dependences but of course rpmbuild on older rh doesn't support them.
I'll propose the Makefile.global patch to hackers too. I'll update if accepted.
On Sun, Mar 17, 2019 at 06:17:55PM +0100, Laurenz Albe wrote: > Justin Pryzby wrote: > > I'm migating a postgres11 server to centos7. > > It turns out we need parray_gin extension. > > > > Under centos6, it compiled and installed fine, but but under c7 it fails: > > > > sudo pgxnclient install parray_gin > > > gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute-Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2-fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic-fPIC -I. -I./ -I/usr/pgsql-11/include/server -I/usr/pgsql-11/include/internal -D_GNU_SOURCE -I/usr/include/libxml2 -I/usr/include -c -o src/parray_gin.o src/parray_gin.c > > > gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute-Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2-fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic-fPIC -I. -I./ -I/usr/pgsql-11/include/server -I/usr/pgsql-11/include/internal -D_GNU_SOURCE -I/usr/include/libxml2 -I/usr/include -c -o src/trgm.o src/trgm.c > > > gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute-Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2-fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic-fPIC -shared -o parray_gin.so src/parray_gin.o src/trgm.o -L/usr/pgsql-11/lib -Wl,--as-needed -L/usr/lib64/llvm5.0/lib -L/usr/lib64 -Wl,--as-needed -Wl,-rpath,'/usr/pgsql-11/lib',--enable-new-dtags > > > cp sql/parray_gin.sql sql/parray_gin--1.3.1.sql > > > /opt/rh/llvm-toolset-7/root/usr/bin/clang -Wno-ignored-attributes -fno-strict-aliasing -fwrapv -O2 -I. -I./ -I/usr/pgsql-11/include/server-I/usr/pgsql-11/include/internal -D_GNU_SOURCE -I/usr/include/libxml2 -I/usr/include -flto=thin-emit-llvm -c -o src/parray_gin.bc src/parray_gin.c > > > gmake: /opt/rh/llvm-toolset-7/root/usr/bin/clang: Command not found > > > gmake: *** [src/parray_gin.bc] Error 127 > > > ERROR: command returned 2: gmake PG_CONFIG=/bin/pg_config all > > If you don't need JIT support for the extension, you could try > > with_llvm=no make -e > > That worked with oracle_fdw. > > Don't know how that translates to pgxnclient though. I ran into this again so reread your mail. It looks like pgxnclient has no nice way to do that, you'd have to create a 1-liner like /usr/local/bin/gmake: #! /bin/sh exec /usr/bin/make -e "$@" And then run: sudo sh -xec 'export with_llvm=no PATH=$PATH:/usr/local/bin:/usr/pgsql-12/bin; pgxnclient install cstore_fdw' Thanks for the idea. -- Justin
Re: Justin Pryzby 2020-04-24 <20200424181956.GQ28974@telsasoft.com> > > If you don't need JIT support for the extension, you could try > > > > with_llvm=no make -e make with_llvm=no should do the same without extra -switches. > sudo sh -xec 'export with_llvm=no PATH=$PATH:/usr/local/bin:/usr/pgsql-12/bin; pgxnclient install cstore_fdw' ... that won't fix that use case, but that can be simplified as well: sudo with_llvm=no PATH=$PATH:/usr/local/bin:/usr/pgsql-12/bin pgxnclient install cstore_fdw Christoph