Обсуждение: Re: RPM source files should be in CVS (was Re: [GENERAL] psql -l)

[cc: to GENERAL replacedby cc: to HACKERS]
On Friday 20 July 2001 17:14, Tom Lane wrote:
> Lamar Owen <lamar.owen@wgcr.org> writes:
> > The biggest patching by far is
> > in the regression tests, which really are not designed to live outside
> > the source tree, but can be munged into shape fairly easily.

> Peter has already done good work in making it possible to build outside
> the source tree.  ISTM that it would make logical sense to allow
> regression tests to be run outside the source tree as well, as long as
> the changes don't break the existing procedures.  I have not looked at
> your patches in this area --- what do they need to do, exactly?

Ok, let's look.  First, there is a createlang issue: during build, @libdir@ 
as referenced in the createlang script references /usr/lib, instead of 
/usr/lib/pgsql, which is desired.  So the first patch is:
diff -uNr postgresql-7.1.2.orig/src/bin/scripts/createlang.sh 
postgresql-7.1.2/src/bin/scripts/createlang.sh
--- postgresql-7.1.2.orig/src/bin/scripts/createlang.sh    Sun Feb 18 13:34:01 
2001
+++ postgresql-7.1.2/src/bin/scripts/createlang.sh    Wed Jun 13 16:00:55 2001
@@ -164,7 +164,7 @@# Check that we have PGLIB# ----------if [ -z "$PGLIB" ]; then
-    PGLIB='@libdir@'
+    PGLIB='/usr/lib/pgsql'fi# ----------

To handle that, as $PGLIB does indeed point to /usr/lib/pgsql for most 
things, but a user is not guaranteed to set the envvar.  @libdir@ points to 
/usr/lib during the build, as it should -- but createlang's PGLIB and 
autoconf's libdir are not equal.  

This is desireable because the procedural languages aren't generally loadable 
into any arbitrary program by ld.so; rather, they are postgresql-specifc 
modules, warranting a separate directory under FHS. This patch fixes the 
RPM-specific case only, obviously, as /usr/lib/pgsql is going to be the wrong 
choice for non-RPM users :-).

Next, we have patches to make the perl client honor RPM_BUILD_ROOT (otherwise 
known as DESTDIR).  I'm omitting them here, as Peter has mentioned a build 
overhaul for the perl and python clients to make them do DESTDIR and in 
general fit in better with the rest of the package.

On to the next batch....  There are a few perl and python scripts shipped as 
examples -- every last one of them shebangs to '/usr/local/perl' or 
'/usr/local/python' -- to make them usable, I patch this to '/usr/bin/perl' 
or python, as appropriate.  I only ship 
postgresql-7.1.2/src/interfaces/perl5/test.pl at this time.

Now to the regression tests.  First off, I:
diff -uNr postgresql-7.1.2.orig/src/test/regress/GNUmakefile 
postgresql-7.1.2/src/test/regress/GNUmakefile
--- postgresql-7.1.2.orig/src/test/regress/GNUmakefile    Wed Apr  4 17:15:56 
2001
+++ postgresql-7.1.2/src/test/regress/GNUmakefile    Wed Jun 13 16:00:55 2001
@@ -67,8 +67,8 @@abs_builddir := $(shell pwd)define sed-command
-sed -e 's,@abs_srcdir@,$(abs_srcdir),g' \
-    -e 's,@abs_builddir@,$(abs_builddir),g' \
+sed -e 's,@abs_srcdir@,/usr/lib/pgsql/test/regress,g' \
+    -e 's,@abs_builddir@,/usr/lib/pgsql/test/regress,g' \    -e 's/@DLSUFFIX@/$(DLSUFFIX)/g' $< >$@endef
since the tests aren't in the build tree anymore, but in 
/usr/lib/pgsql/test/regress.  Well _technically_ they're really NOT in 
/usr/lib/pgsql/test/regress, but in DESTDIR/usr/lib/pgsql/test/regress during 
the build -- but they will be executed in the coded location after the RPM 
installation.

Then, I:
-   AS '@abs_builddir@/regress@DLSUFFIX@'
+   AS '/usr/lib/pgsql/test/regress/regress.so'
everywhere that is used, along with its likenesses pointing to refint.so and 
autoinc.so, which I prebuild and stuff into /usr/lib/pgsql/test/regress.  
Although /usr/lib/pgsql would be a more consistent place, I guess.  That 
construct is used in 
postgresql-7.1.2/src/test/regress/input/create_function_1.source and 
ostgresql-7.1.2/src/test/regress/output/create_function_1.source.

Finally, I patch postgresql-7.1.2/src/test/regress/pg_regress.sh:
@@ -69,7 +69,7 @@: ${inputdir=.}: ${outputdir=.}
-libdir='@libdir@'
+libdir='/usr/lib/pgsql'bindir='@bindir@'datadir='@datadir@'host_platform='@host_tuple@'

Again, @libdir@ != $PGLIB.

This set is quite a bit smaller than the 7.0.x and 6.5.x sets, thanks in no 
small part to Peter's work, as you have already said.

> I think there are really two separate discussions going on here: one is
> whether we shouldn't try harder to roll some of the RPMset diffs back
> into the main sources, and the other is how we can make information
> about some of the popular packages more readily visible/available to the
> developers.

My diffs are nowhere near as large as the debian set.  There are other things 
I could patch, instead of frobbing in the specfile, though -- things like the 
python and perl clients' makefile's DESTDIR ignorance, and the fact that 
'make install' puts the procedural languages in /usr/lib instead of 
/usr/lib/pgsql.  The easy answer: 'Use the --libdir configure switch!' won't 
work, though, as I can't just tell configure that libdir needs to be 
/usr/lib/pgsql -- libpq.so and friends need to go in /usr/lib!

And I've not tried to make my patches fit the general case as yet -- they 
haven't needed to be general in scope.

There is some munging done in contrib that could be put in a patch, though -- 
in particular, the following construct executes _14_ times....
# some-contrib-module
pushd some-contrib-module
perl -pi -e "s|/usr/lib/contrib|/usr/lib/pgsql/contrib/some-contrib-module|" *
popd

libdir !=$PGLIB, again.

And more path munging for the rserv and earthdistance contrib modules happen.

And the whole contrib tree, since there isn't a good 'make install' that 
honors DESTDIR for that tree, gets a kick in the pants to 
/usr/lib/pgsql/contrib, from $RPM_BUILD_DIR/postgresql-7.1.2.

> Peter's stance on the latter seems to be "go look at the
> packagers' sites", which is defensible, but that's the current approach
> and I think it's not working. 

The biggest RPM difference is simply where things are put.  Otherwise there's 
a mere handful of sysadmin scripts added.

With the specfile, README.rpm-dist, and the other scriptfiles in a CVS 
module, I'd sleep better, knowing that someone else might have an easier time 
picking things up if I kick the big bucket.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Lamar Owen <lamar.owen@wgcr.org> writes:
> Ok, let's look.  First, there is a createlang issue: during build, @libdir@ 
> as referenced in the createlang script references /usr/lib, instead of 
> /usr/lib/pgsql, which is desired.

Okay, that problem is gone in current sources, anyway (createlang no
longer needs to know any absolute paths).

> On to the next batch....  There are a few perl and python scripts shipped as 
> examples -- every last one of them shebangs to '/usr/local/perl' or 
> '/usr/local/python' -- to make them usable, I patch this to '/usr/bin/perl' 
> or python, as appropriate.

Hmm.  Given that they're only examples, and are clearly going to be
broken until hand-edited on many systems not only RedHat, it's not clear
that this is worth your worrying about.  But by the same token, I
wouldn't have a problem with applying that change to the masters ---
surely there are as many systems where '/usr/bin/perl' is correct as
there are where the other is correct.  (In fact, a quick grep shows that
we have more '/usr/bin/perl' than '/usr/local/bin/perl' in the
distribution, so your claim that they're all the latter is mistaken.
We should certainly try to make them consistent, whichever is
preferred.)

BTW, the only python shebangs I can find in CVS look like#! /usr/bin/env python
Isn't that OK on RedHat?

> Now to the regression tests.  First off, I:
>  define sed-command
> -sed -e 's,@abs_srcdir@,$(abs_srcdir),g' \
> -    -e 's,@abs_builddir@,$(abs_builddir),g' \
> +sed -e 's,@abs_srcdir@,/usr/lib/pgsql/test/regress,g' \
> +    -e 's,@abs_builddir@,/usr/lib/pgsql/test/regress,g' \
>      -e 's/@DLSUFFIX@/$(DLSUFFIX)/g' $< >$@
>  endef

Clearly, this needs to be generalized ...

> Then, I:
> -   AS '@abs_builddir@/regress@DLSUFFIX@'
> +   AS '/usr/lib/pgsql/test/regress/regress.so'
> everywhere that is used, along with its likenesses pointing to refint.so and 
> autoinc.so, which I prebuild and stuff into /usr/lib/pgsql/test/regress.  

Much of this could be eliminated given the new path-searching behavior
for CREATE FUNCTION, I think.  Actually I thought Peter had cleaned it
up already, but I see he hasn't touched the regression tests.  IMHO we
could have "make installcheck" copy the .so files to $LIBDIR, and then
the regression test input and output files themselves wouldn't need to
know these paths at all.  (OTOH, there'd still be paths in the COPY
commands.  Would it be okay to eliminate testing of backend COPY and
instead make these regression tests use psql \copy?)
        regards, tom lane

Tom Lane <tgl@sss.pgh.pa.us> writes:

> BTW, the only python shebangs I can find in CVS look like
>     #! /usr/bin/env python
> Isn't that OK on RedHat?

It is.

-- 
Trond Eivind Glomsrød
Red Hat, Inc.

On Fri, Jul 20, 2001 at 07:05:46PM -0400, Trond Eivind Glomsr?d wrote:
> Tom Lane <tgl@sss.pgh.pa.us> writes:
> 
> > BTW, the only python shebangs I can find in CVS look like
> >     #! /usr/bin/env python
> > Isn't that OK on RedHat?
> 
> It is.

Probably the perl scripts should say, likewise, 
 #!/usr/bin/env perl

Nathan Myers
ncm@zembu.com

On Friday 20 July 2001 18:45, Tom Lane wrote:
> Lamar Owen <lamar.owen@wgcr.org> writes:
> > On to the next batch....  There are a few perl and python scripts shipped
> > as examples -- every last one of them shebangs to '/usr/local/perl' or
> > '/usr/local/python' -- to make them usable, I patch this to
> > '/usr/bin/perl' or python, as appropriate.

> Hmm.  Given that they're only examples, and are clearly going to be
> broken until hand-edited on many systems not only RedHat, it's not clear

Well, there were more than just a few at one point.  In any case, it's been 
awhile since I combed through the example scripts -- of which I only now ship 
the one, which is designed to test the perl client -- which I find to be a 
useful thing.

> BTW, the only python shebangs I can find in CVS look like
>     #! /usr/bin/env python
> Isn't that OK on RedHat?

Yeah, that construct is OK.  7.0.x was different, unless I'm far off-base.  
But I'm not shipping any patched python scripts with 7.1.x anyway -- the 
6.5.x and 7.0.x dists had some scripts with #!/usr/local/bin/python.

So much for my 'every last one,' eh? :-)

> Much of this could be eliminated given the new path-searching behavior
> for CREATE FUNCTION, I think.  Actually I thought Peter had cleaned it
> up already, but I see he hasn't touched the regression tests.  

How is this search path defined?  Blindly using libdir is not ok -- 
libdir!=PGLIB, and PGLIB may not be defined in the environment -- it might be 
there, but we can't count on it.

> IMHO we
> could have "make installcheck" copy the .so files to $LIBDIR,

libdir!=PGLIB for the RPMs.  libdir=/usr/lib; PGLIB=/usr/lib/pgsql.  I was so 
happy when the bki sources were no longer referenced by PGLIB -- when the 
procedural language handlers aren't thusly referenced will be a Happy Day.  
If PGLIB could = libdir, and something like PGHANDLER= where the handlers 
live, I'd also be happy.  If this function search path can be configured to 
search in /usr/lib/pgsql and all or any of its subs, while libpq and kin live 
in /usr/lib, I _will_ be happy.

> and then
> the regression test input and output files themselves wouldn't need to
> know these paths at all.  (OTOH, there'd still be paths in the COPY
> commands.  Would it be okay to eliminate testing of backend COPY and
> instead make these regression tests use psql \copy?)

The COPY paths are munged into form by the GNUmakefile patch -- so, if the 
GNUmakefile can generally deal with the paths by placing relative paths 
(relative to what, though?) in the @abs_srcdir@/@abs_builddir@ substitutions, 
then those paths aren't an issue.

Although a psql \copy regression test might be a good thing in its own right.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Lamar Owen <lamar.owen@wgcr.org> writes:
> How is this search path defined?  Blindly using libdir is not ok -- 

Why not?

The search path is defined in postgresql.conf (and I see Peter forgot
to add an example to the postgresql.conf.sample file), but the default
is the backend-compile-time $libdir.  Offhand I don't see what's wrong
with it.

> libdir!=PGLIB, and PGLIB may not be defined in the environment -- it
> might be there, but we can't count on it.

AFAICT we do not depend on environment PGLIB any more.  Configure-time
$libdir is what counts.

> If this function search path can be configured to search in
> /usr/lib/pgsql and all or any of its subs, while libpq and kin live in
> /usr/lib, I _will_ be happy.

I think all you need to do is set up postgresql.conf that way.
        regards, tom lane

> > On to the next batch....  There are a few perl and python scripts shipped as
> > examples -- every last one of them shebangs to '/usr/local/perl' or
> > '/usr/local/python' -- to make them usable, I patch this to '/usr/bin/perl'
> > or python, as appropriate.
> Hmm.  Given that they're only examples, and are clearly going to be
> broken until hand-edited on many systems not only RedHat, it's not clear
> that this is worth your worrying about.

Ack! There is a way to write this stuff to be portable. The tricks
change a bit depending on the scripting language you are using, but for
perl this is how the header should look:

#!/bin/sh
# -*- perl -*-
# the line above helps with emacs, and put other comments here...

eval '(exit $?0)' && eval 'exec perl -S $0 ${1+"$@"}'   & eval 'exec perl -S $0 $argv:q'   if 0;

# real perl code follows...


There is no reason to have a dependency on anything but the location of
sh, which is much more reliable than locations for perl, tcl, etc etc.
Not sure the exact form of this technique for python (maybe the same as
above) but there is a similar but not identical form for tcl code
(examples available on request; the above for perl is demonstrated in
contrib/rserv/*.in).
                        - Thomas

On Friday 20 July 2001 19:56, Tom Lane wrote:
> Lamar Owen <lamar.owen@wgcr.org> writes:
> > How is this search path defined?  Blindly using libdir is not ok --

> Why not?

During RPM build, libdir will point to /usr/lib.  This is OK and appropriate 
for the generally-loadable shared libs.  HOWEVER, the shared objects that are 
affected by this function handler load search path will not be located in 
/usr/lib -- as they are not generally loadable shared libraries, but 
PostgreSQL-only shared libraries.  Thus, /usr/lib/pgsql is the home for 
those, and, in an RPM installation, should be the head of any shared object 
load path searched by the PostgreSQL dynaloader.

/usr/lib -> system dynaloader default for non-essentials (/lib of course for 
libs essential for boot)
/usr/lib/pgsql -> PostgreSQL dynaloader default (ideally, on an FHS-compliant 
installation).

> The search path is defined in postgresql.conf (and I see Peter forgot
> to add an example to the postgresql.conf.sample file), but the default
> is the backend-compile-time $libdir.  Offhand I don't see what's wrong
> with it.

I can patch postgresql.conf.sample easily enough -- but, if I'm trying to get 
away from RPM-specific patches.... :-) I have heretofore not modified the 
default postgresql.conf in anyway -- no tuning, no tcpip_socket setting (for 
which I get some grief, as people running  RPMs of 7.0.x are used to -i being 
on by default), no nothing.

Not knowing alot about autoconf, I hesitate to suggest or ask the following, 
but I will anyway -- is it possible to define an optional 
'so-search-path-default' switch for the backend's dynaloader?  This has 
nothing to do with the OS dynaloader path, for which there is a well-defined 
config file on most OS's -- just the backend's function handler dynaloader.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11