Обсуждение: Int64GetDatum
I have compiled some C (pljava.c) for solaris sparc 64 bit, setup the LD_LIBRARY_PATH so postgres can find it, and try and load it. me=# CREATE FUNCTION sqlj.java_call_handler() RETURNS language_handler AS 'pljava' LANGUAGE C; ERROR: could not load library "/opt/mystuff/pljava/pljava.so": ld.so.1: postgres: fatal: relocation error: file /opt/mystuff/pljava/pljava.so: symbol Int64GetDatum: referenced symbol not found oops. anyways, some digging (thank you kris jurka!) has shown that my 64bit solaris tarball (from the postgres website) is missing the 64 bit includes and lib/64/pgxs/src stuff. for various reasons (many political) I'd like to get my module working with this version, hence would like to find the includes and stuff that were built with it. the website says Bjorn Munch of Sun built them. Does he monitor this list?
John R Pierce <pierce@hogranch.com> writes:
> I have compiled some C (pljava.c) for solaris sparc 64 bit, setup the
> LD_LIBRARY_PATH so postgres can find it, and try and load it.
> me=# CREATE FUNCTION sqlj.java_call_handler() RETURNS language_handler
> AS 'pljava' LANGUAGE C;
> ERROR: could not load library "/opt/mystuff/pljava/pljava.so": ld.so.1:
> postgres: fatal: relocation error: file /opt/mystuff/pljava/pljava.so:
> symbol Int64GetDatum: referenced symbol not found
This appears to be a consequence of 32-vs-64-bit confusion, ie, your
pljava.so was built assuming !USE_FLOAT8_BYVAL but the backend was
built with that symbol defined. Are you still trying to hack your
way to a solution without making configure run properly? Because
this is just about the sort of pain I'd expect from that "shortcut".
regards, tom lane
Tom Lane wrote: > John R Pierce <pierce@hogranch.com> writes: > >> I have compiled some C (pljava.c) for solaris sparc 64 bit, setup the >> LD_LIBRARY_PATH so postgres can find it, and try and load it. >> > > >> me=# CREATE FUNCTION sqlj.java_call_handler() RETURNS language_handler >> AS 'pljava' LANGUAGE C; >> ERROR: could not load library "/opt/mystuff/pljava/pljava.so": ld.so.1: >> postgres: fatal: relocation error: file /opt/mystuff/pljava/pljava.so: >> symbol Int64GetDatum: referenced symbol not found >> > > This appears to be a consequence of 32-vs-64-bit confusion, ie, your > pljava.so was built assuming !USE_FLOAT8_BYVAL but the backend was > built with that symbol defined. Are you still trying to hack your > way to a solution without making configure run properly? Because > this is just about the sort of pain I'd expect from that "shortcut". > Using the include files provided with the 64bit version is giving me the wrong Float8 type, yes, as they are the 32bit include files. I need to build pl/java to run against the binary release of Postgres for largely political/corporate reasons. this is to be installable as an addon to an existing large/complex database deployment. If I build my own ./configure and compile my own postgres binary and use that to build this, what are the odds the module I compile will work with the released binary? If I had the correct include files for the binary release, I would not be having this problem.
John R Pierce <pierce@hogranch.com> writes:
> Using the include files provided with the 64bit version is giving me the
> wrong Float8 type, yes, as they are the 32bit include files.
> I need to build pl/java to run against the binary release of Postgres
> for largely political/corporate reasons. this is to be installable as
> an addon to an existing large/complex database deployment.
Well, in that case you'd better pester whoever packages the binary
release you're using to ship a consistent set of files. What you
have is a broken package, IMNSHO.
regards, tom lane
Tom Lane wrote: > John R Pierce <pierce@hogranch.com> writes: > >> I need to build pl/java to run against the binary release of Postgres >> for largely political/corporate reasons. this is to be installable as >> an addon to an existing large/complex database deployment. >> > > Well, in that case you'd better pester whoever packages the binary > release you're using to ship a consistent set of files. What you > have is a broken package, IMNSHO. > If I were John, I'd be preparing to dig in on providing a complete source build with PL/Java installed. It looks like the idea that they'll be able to take their *existing* Solaris binaries and just add Java on top of them is going to end up more risky than doing that. The best approach for this situation as far as I'm concerned is a build to a completely alternate location, not even touching the system PostgreSQL. Then you can slide the new version onto there without touching the known working one at all, just swap the paths around--and rollback is just as easy. I think the road these bad corporate mandates is leading toward is one where you risk breaking the system database install by hacking it up without a good rollback position in case of an unexpected problem. That's something they'd get for free if they just accepted that the whole database binary set needs to be replaced. Would have been different if the Java just compiled and worked, but there's obviously some deeper issues here. As Tom points out, the only solution there might end up being a new binary set from the packager, which means a wholesale swap of binaries anyway. People should not make decisions about large/complex database environments for political/corporate reasons when those reasons end up increasing the risk of problems. If you get pushed around that way regardless, a "I told you so" CYA memo suggesting as much would be appropriate here, so you don't get blamed for any fallout. -- Greg Smith 2ndQuadrant US Baltimore, MD PostgreSQL Training, Services and Support greg@2ndQuadrant.com www.2ndQuadrant.us
Greg Smith wrote: > If I were John, I'd be preparing to dig in on providing a complete > source build with PL/Java installed. It looks like the idea that > they'll be able to take their *existing* Solaris binaries and just add > Java on top of them is going to end up more risky than doing that. > The best approach for this situation as far as I'm concerned is a > build to a completely alternate location, not even touching the system > PostgreSQL. Then you can slide the new version onto there without > touching the known working one at all, just swap the paths around--and > rollback is just as easy. so you're saying that building plugins to work with an existing system is bad? then whats the point of the whole pgxs system and including server headers in a binary release? compiling a plugin like pl/java makes no reference to the source tree at all, rather, it uses the lib/pgxs/src/Makefile.global and include/server/*.h which are part of a binary release. I'm simply dealing with a situation here where the packager of the Solaris binary didn't realize those files varied between 32 and 64, and neglected to include the right ones in the 64bit build. He's popped up on hackers, and is looking into it now. fyi, the packages in question are the 8.4.x ones here, http://www.postgresql.org/download/solaris as well as the ones provided by Sun (which I believe come from the same packager)
John R Pierce <pierce@hogranch.com> writes:
> Greg Smith wrote:
>> If I were John, I'd be preparing to dig in on providing a complete
>> source build with PL/Java installed. It looks like the idea that
>> they'll be able to take their *existing* Solaris binaries and just add
>> Java on top of them is going to end up more risky than doing that.
>> The best approach for this situation as far as I'm concerned is a
>> build to a completely alternate location, not even touching the system
>> PostgreSQL. Then you can slide the new version onto there without
>> touching the known working one at all, just swap the paths around--and
>> rollback is just as easy.
> so you're saying that building plugins to work with an existing system
> is bad?
No, but trying to build against a non-self-consistent set of files is
bad. You really need a pg_config.h that matches the original build of
the server, and you haven't got that. I think Greg's point is that
trying to reverse-engineer that file is considerably more risky than
building your own packages from scratch.
> I'm simply dealing with a situation here where the packager of the
> Solaris binary didn't realize those files varied between 32 and 64, and
> neglected to include the right ones in the 64bit build. He's popped up
> on hackers, and is looking into it now.
Right. If you can get a consistent fileset from Bjorn in a timely
fashion, problem solved.
regards, tom lane
John R Pierce wrote: > so you're saying that building plugins to work with an existing system > is bad? then whats the point of the whole pgxs system and including > server headers in a binary release? It's fine if your package has been setup to allow it. I bundle up stuff on RHEL like that all the time, works fine. What I'm suggesting is that as soon as that assumption is proven to be false, which is the case for you, the most straightforward way to make sure you have a consistent database+extension set is to build the whole stack together yourself. If you can get this resolved via Bjorn fast enough for you, that may turn out to be fine. But recognize that even in that case, you may end up swapping around files related to the basic server portion of your production system to pull those fixes in--exactly the thing you are suggesting is off limits. Once you've demonstrated that the files on the server already really aren't suitable for the job you need them for, this idea that they're the sacred set that must not be touched is busted anyway. At that point, I start thinking in terms of how I'm going to test this out and how rollback in case of a problem is going to work, and that can turn to be harder to handle with a change to the system database installation than via running your own set of binaries in parallel with it. I'm not trying to criticize what you're doing, just given you a dose of my own paranoia and preferred risk management approach for this sort of thing. It may not actually be possible to fully follow the unreasonable requirements you've been given and deliver something that works well. -- Greg Smith 2ndQuadrant US Baltimore, MD PostgreSQL Training, Services and Support greg@2ndQuadrant.com www.2ndQuadrant.us
Tom Lane wrote:
> Right. If you can get a consistent fileset from Bjorn in a timely
> fashion, problem solved.
>
exactly. that is my intent. Bjorn replied to my request on hackers
last night, and 'is going to look into it'
can someone confirm, the critical files that get customized by
./configure are
$INCLUDEDIR/pg_config.h
$INCLUDEDIR/server/pg_config.h (apparently identical)
$LIBDIR/pgxs/src/Makefile.global
are there others which change that would make it into a binary release?
Greg Smith wrote: > I'm not trying to criticize what you're doing, just given you a dose > of my own paranoia and preferred risk management approach for this > sort of thing. It may not actually be possible to fully follow the > unreasonable requirements you've been given and deliver something that > works well. > understood. and if it was my server, no problem. but I work for a development group stateside within my company, these deployed servers are at our Asian manufacturing plants, and the production operations people are rather far removed from me. they are also hyper conservative (someone asks for a presentation on Cloud architectures, they came up with a slide set on IBM Power Server LPAR's and called that a cloud!)
John R Pierce <pierce@hogranch.com> writes:
> can someone confirm, the critical files that get customized by
> ./configure are
> $INCLUDEDIR/pg_config.h
> $INCLUDEDIR/server/pg_config.h (apparently identical)
> $LIBDIR/pgxs/src/Makefile.global
I believe all of the files that get written at the end of configure are
potentially architecture-sensitive. There are at least two other .h
files that qualify: ecpg_config.h and pg_config_os.h (the latter is a
symlink). A quick look through configure.in shows these files getting
made:
AC_CONFIG_LINKS([src/backend/port/tas.s:src/backend/port/tas/${tas_file}])
AC_CONFIG_FILES([GNUmakefile src/Makefile.global])
AC_CONFIG_LINKS([
src/backend/port/dynloader.c:src/backend/port/dynloader/${template}.c
src/backend/port/pg_sema.c:${SEMA_IMPLEMENTATION}
src/backend/port/pg_shmem.c:${SHMEM_IMPLEMENTATION}
src/include/dynloader.h:src/backend/port/dynloader/${template}.h
src/include/pg_config_os.h:src/include/port/${template}.h
src/Makefile.port:src/makefiles/Makefile.${template}
])
AC_CONFIG_HEADERS([src/include/pg_config.h],
[
# Update timestamp for pg_config.h (see Makefile.global)
echo >src/include/stamp-h
])
AC_CONFIG_HEADERS([src/interfaces/ecpg/include/ecpg_config.h],
[echo >src/interfaces/ecpg/include/stamp-h])
The .c files are probably not relevant for building add-on modules, but
I think all the others are.
regards, tom lane
Tom Lane wrote: > No, but trying to build against a non-self-consistent set of files is > bad. You really need a pg_config.h that matches the original build of > the server, and you haven't got that. I think Greg's point is that > trying to reverse-engineer that file is considerably more risky than > building your own packages from scratch. > and once again totally concurring with what you are saying... for laughs, I went ahead and tweaked the fields in a copy of the 32bit pg_config.h that I suspected were obviously wrong, and rebuilt my module against that, and it got through most of the pl/java test suite. so I know I'm on the right track, I just need the right pg_config.h from Bjorn and all will be well with my world.
John R Pierce wrote: > Tom Lane wrote: > > No, but trying to build against a non-self-consistent set of files is > > bad. You really need a pg_config.h that matches the original build of > > the server, and you haven't got that. I think Greg's point is that > > trying to reverse-engineer that file is considerably more risky than > > building your own packages from scratch. > > > and once again totally concurring with what you are saying... for > laughs, I went ahead and tweaked the fields in a copy of the 32bit > pg_config.h that I suspected were obviously wrong, and rebuilt my module > against that, and it got through most of the pl/java test suite. > > > so I know I'm on the right track, I just need the right pg_config.h from > Bjorn and all will be well with my world. Yes, great. One point is that while you are trying to fix this for the one-off case, we should be realizing that we need a proper fix so all your future upgrades will be clean, and other users will not also have this problem. I agree with your approach to first find out if the Solaris build is wrong, and then get that fixed. You are right that just rebuilding the install for pl/java would not have accomplished the larger fix. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com
Bruce Momjian wrote: > Yes, great. One point is that while you are trying to fix this for > the one-off case, we should be realizing that we need a proper fix so > all your future upgrades will be clean, and other users will not also > have this problem. I agree with your approach to first find out if the > Solaris build is wrong, and then get that fixed. You are right that > just rebuilding the install for pl/java would not have accomplished the > larger fix. > yeah, for sure. there's a definite structural problem in that the 32 bit and 64 bit Solaris binaries are in the same tree, with just the bin and lib directories differentiated. you can in theory install them both, and point them to different $PGDATA directories, and have them both running at once, but they share the include directory, so anything like a pl/*** you build from source against this tree will have problems. A temporary acceptable solution would be to simply acknowlege this problem in the build release notes and specify that you either only install the one package you want, or if you install them both, the one you install *last* will be the only one you can build something like pl/java against. This is somewhat sketchy, but still an improvement over the current. I don't know if the build trees can be structured so the include directories can be differentiated the same as the bin and lib... I see the include dir location is supplied by pg_config, but I don't know how consistently this is used by something like pl/java's build. # /usr/postgres/8.4-community/bin/pg_config # 32bit version BINDIR = /usr/postgres/8.4-community/bin DOCDIR = /usr/postgres/8.4-community/share/doc HTMLDIR = /usr/postgres/8.4-community/share/doc INCLUDEDIR = /usr/postgres/8.4-community/include PKGINCLUDEDIR = /usr/postgres/8.4-community/include INCLUDEDIR-SERVER = /usr/postgres/8.4-community/include/server LIBDIR = /usr/postgres/8.4-community/lib PKGLIBDIR = /usr/postgres/8.4-community/lib LOCALEDIR = /usr/postgres/8.4-community/share/locale MANDIR = /usr/postgres/8.4-community/man SHAREDIR = /usr/postgres/8.4-community/share SYSCONFDIR = /usr/postgres/8.4-community/etc PGXS = /usr/postgres/8.4-community/lib/pgxs/src/makefiles/pgxs.mk .......rest deleted..... # /usr/postgres/8.4-community/bin/64/pg_config # 64bit version BINDIR = /usr/postgres/8.4-community/bin/64 DOCDIR = /usr/postgres/8.4-community/share/doc HTMLDIR = /usr/postgres/8.4-community/share/doc INCLUDEDIR = /usr/postgres/8.4-community/include PKGINCLUDEDIR = /usr/postgres/8.4-community/include INCLUDEDIR-SERVER = /usr/postgres/8.4-community/include/server LIBDIR = /usr/postgres/8.4-community/lib/64 PKGLIBDIR = /usr/postgres/8.4-community/lib/64 LOCALEDIR = /usr/postgres/8.4-community/share/locale MANDIR = /usr/postgres/8.4-community/man SHAREDIR = /usr/postgres/8.4-community/share SYSCONFDIR = /usr/postgres/8.4-community/etc PGXS = /usr/postgres/8.4-community/lib/64/pgxs/src/makefiles/pgxs.mk .......rest deleted.....
John R Pierce <pierce@hogranch.com> writes:
> I don't know if the build trees can be structured so the include
> directories can be differentiated the same as the bin and lib...
The RPM distributions are able to deal with this without actually
differentiating: if you install both 32- and 64-bit RPMs then the
overlapping files are the same for both, with pg_config.h being
a stub that does this:
#if defined(__x86_64__)
#include "pg_config_x86_64.h"
#elif defined(__i386__)
#include "pg_config_i386.h"
... etc ...
I don't know if Solaris' package management offers the same ability, but
if it does, then it could be handled like that.
regards, tom lane
John R Pierce wrote: > there's a definite structural problem in that the 32 bit and 64 bit > Solaris binaries are in the same tree, with just the bin and lib > directories differentiated. you can in theory install them both, and > point them to different $PGDATA directories, and have them both running > at once, but they share the include directory, so anything like a > pl/*** you build from source against this tree will have problems. > > A temporary acceptable solution would be to simply acknowlege this > problem in the build release notes and specify that you either only > install the one package you want, or if you install them both, the one > you install *last* will be the only one you can build something like > pl/java against. This is somewhat sketchy, but still an > improvement over the current. > > I don't know if the build trees can be structured so the include > directories can be differentiated the same as the bin and lib... I see > the include dir location is supplied by pg_config, but I don't know how > consistently this is used by something like pl/java's build. > > # /usr/postgres/8.4-community/bin/pg_config # 32bit version > BINDIR = /usr/postgres/8.4-community/bin > DOCDIR = /usr/postgres/8.4-community/share/doc > HTMLDIR = /usr/postgres/8.4-community/share/doc > INCLUDEDIR = /usr/postgres/8.4-community/include > PKGINCLUDEDIR = /usr/postgres/8.4-community/include > INCLUDEDIR-SERVER = /usr/postgres/8.4-community/include/server > LIBDIR = /usr/postgres/8.4-community/lib > PKGLIBDIR = /usr/postgres/8.4-community/lib > LOCALEDIR = /usr/postgres/8.4-community/share/locale > MANDIR = /usr/postgres/8.4-community/man > SHAREDIR = /usr/postgres/8.4-community/share > SYSCONFDIR = /usr/postgres/8.4-community/etc > PGXS = /usr/postgres/8.4-community/lib/pgxs/src/makefiles/pgxs.mk > .......rest deleted..... > > # /usr/postgres/8.4-community/bin/64/pg_config # 64bit version > BINDIR = /usr/postgres/8.4-community/bin/64 > DOCDIR = /usr/postgres/8.4-community/share/doc > HTMLDIR = /usr/postgres/8.4-community/share/doc > INCLUDEDIR = /usr/postgres/8.4-community/include > PKGINCLUDEDIR = /usr/postgres/8.4-community/include > INCLUDEDIR-SERVER = /usr/postgres/8.4-community/include/server > LIBDIR = /usr/postgres/8.4-community/lib/64 > PKGLIBDIR = /usr/postgres/8.4-community/lib/64 > LOCALEDIR = /usr/postgres/8.4-community/share/locale > MANDIR = /usr/postgres/8.4-community/man > SHAREDIR = /usr/postgres/8.4-community/share > SYSCONFDIR = /usr/postgres/8.4-community/etc > PGXS = /usr/postgres/8.4-community/lib/64/pgxs/src/makefiles/pgxs.mk > .......rest deleted..... OK, I see your bin and lib 64-bit files are places into a "64" subdirectory. Logic dictates that the include files should be in /usr/postgres/8.4-community/include/64, and /usr/postgres/8.4-community/bin/64/pg_config should report that location. If it does, all the PGXS stuff will use it. I don't think we ever considered this kind of setup, but that seem to be the proper direction. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com
Tom Lane wrote: > John R Pierce <pierce@hogranch.com> writes: > > I don't know if the build trees can be structured so the include > > directories can be differentiated the same as the bin and lib... > > The RPM distributions are able to deal with this without actually > differentiating: if you install both 32- and 64-bit RPMs then the > overlapping files are the same for both, with pg_config.h being > a stub that does this: > > #if defined(__x86_64__) > #include "pg_config_x86_64.h" > #elif defined(__i386__) > #include "pg_config_i386.h" > ... etc ... > > I don't know if Solaris' package management offers the same ability, but > if it does, then it could be handled like that. Ah, good idea. Is pg_config.h the only file that varies from 32 to 64-bit? -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com
Bruce Momjian wrote: > Tom Lane wrote: > >> John R Pierce <pierce@hogranch.com> writes: >> >>> I don't know if the build trees can be structured so the include >>> directories can be differentiated the same as the bin and lib... >>> >> The RPM distributions are able to deal with this without actually >> differentiating: if you install both 32- and 64-bit RPMs then the >> overlapping files are the same for both, with pg_config.h being >> a stub that does this: >> >> #if defined(__x86_64__) >> #include "pg_config_x86_64.h" >> #elif defined(__i386__) >> #include "pg_config_i386.h" >> ... etc ... >> >> I don't know if Solaris' package management offers the same ability, but >> if it does, then it could be handled like that. >> > > Ah, good idea. Is pg_config.h the only file that varies from 32 to > 64-bit? > > it *appears* to be here, but I don't have the 64 bit file set to compare with. there are definately a couple other files that get generated by ./configure, including ecpg_config.h and another that escapes me at the moment. from what all I could tell, nothing in that would be likely to change between 32 and 64 bit builds on either sparc or x86 (but they might vary between sparc and x86, or even between completely different PG builds, like integer vs numeric date)
John R Pierce <pierce@hogranch.com> writes:
> Bruce Momjian wrote:
>> Ah, good idea. Is pg_config.h the only file that varies from 32 to
>> 64-bit?
> it *appears* to be here, but I don't have the 64 bit file set to compare
> with. there are definately a couple other files that get generated by
> ./configure, including ecpg_config.h and another that escapes me at the
> moment.
ecpg_config.h also has to be stubbed to make this work. The other stuff
that could vary (like Makefile.global) gets installed under bin/ or lib/
so if you have a lib-vs-lib64 type of layout then that's safe.
regards, tom lane
John R Pierce wrote: > I have compiled some C (pljava.c) for solaris sparc 64 bit, setup the > LD_LIBRARY_PATH so postgres can find it, and try and load it. > > me=# CREATE FUNCTION sqlj.java_call_handler() RETURNS > language_handler AS 'pljava' LANGUAGE C; > ERROR: could not load library "/opt/mystuff/pljava/pljava.so": > ld.so.1: postgres: fatal: relocation error: file > /opt/mystuff/pljava/pljava.so: symbol Int64GetDatum: referenced symbol > not found > > oops. > anyways, some digging (thank you kris jurka!) has shown that my 64bit > solaris tarball (from the postgres website) is missing the 64 bit > includes and lib/64/pgxs/src stuff. for various reasons (many > political) I'd like to get my module working with this version, hence > would like to find the includes and stuff that were built with it. the > website says Bjorn Munch of Sun built them. Does he monitor this list? following up to my own post (how RUDE!).... Bjorn came through with the pg_config.h and Makefile.global from his 64bit build, I dropped those in place on my dev box, and voila, I have pl/java working, and passing all the test suite except one test of javaU untrusted, which for now I'm chalking up to gremlins.