Обсуждение: build error: strlcat/strlcpy used from heimdal libroken.so

Dear maintainers of PostgreSQL,

Build errors for PostgreSQL have been reported on Gentoo Linux, see
https://bugs.gentoo.org/285953 for the full bug report there.

The issue has originally been reported for PostgreSQL 8.4.1, but has
been reproduced with 8.4.2, 8.4.3 and even some 9.0 alpha.

In short, the build fails when the heimdal kerberos implementation is
installed on the system, and heimdal wasn't built with -Wl,--as-needed.

In that case, the postgresql configure script detects that the strlcat
and strlcpy functions as available. It does so because libgssapi.so and
libkrb5.so both depend on heimdal's libroken.so, which currently does
export these symbols. A fix for heimdal is in progress, which will
reduce the list of exported symbols using a linker script, but afaik it
hasn't been released yet, so for now heimdal installations providing
these symbols are a fact of life.

Later on, when postgresql has been configured and gets compiled, the
-Wl,--as-needed flag is passed to the linker, and as libroken itself
isn't named as a library to link against, the linker will complain about
undefined references.

Parts of the configure output:
> checking whether strlcat is declared... no
> checking whether strlcpy is declared... no
> checking for strlcat... yes
> checking for strlcpy... yes

Parts of the make output:
> descriptor.o: In function `descriptor_variable':
> .../postgresql-8.4.1/src/interfaces/ecpg/preproc/descriptor.c:326: undefi=
ned reference to `strlcpy'
> ../../../../src/port/libpgport.a(path.o): In function `get_home_path':
> .../postgresql-8.4.1/src/port/path.c:644: undefined reference to `strlcpy'
> ../../../../src/port/libpgport.a(path.o): In function `join_path_componen=
ts':
> .../postgresql-8.4.1/src/port/path.c:177: undefined reference to `strlcpy'
> ../../../../src/port/libpgport.a(path.o): In function `make_relative_path=
':
> .../postgresql-8.4.1/src/port/path.c:503: undefined reference to `strlcpy'
> .../postgresql-8.4.1/src/port/path.c:523: undefined reference to `strlcpy'
> collect2: ld returned 1 exit status
> make[4]: *** [ecpg] Error 1

Several solutions are conceivable. The Gentoo user Honza Mach=C3=A1=C4=8Dek
proposed handling libroken the same way you already handle libedit, i.e.
disable some libraries during detection of these functions. See
https://bugs.gentoo.org/attachment.cgi?id=3D214023 for his patch. In short
it's this:
> -# Some versions of libedit contain strlcpy(); so disregard that library =
while
> +# Some versions of libedit and libroken contain strlcpy(); so disregard =
that library while
>  # checking for these standard libc functions.
>  pgac_save_LIBS=3D"$LIBS"
> -LIBS=3D`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
> +LIBS=3D`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g' -e 's/-=
lgssapi//g' -e 's/-lkrb5//g'`

Another alternative might be choosing behaviour based on the detection
of suitable function declarations, i.e. HAVE_DECL_STRLCAT instead of
HAVE_STRLCAT. Or you might try enabling the -Wl,--as-needed flag before
detecting these library functions, to ensure the same behaviour at
configure time and at build time. In both cases you might whish to
rename those symbols in case you use your own implementation, just to
make sure the linker knows which one to choose.

Please choose a suitable method to fix this build error for PostgreSQL.

Thanks,
 Martin von Gagern

I haven't seen anyone reply to this report, which might mean we are
waiting for Gentoo to be fixed.  Gentoo is notoriously dynamic and it is
hard for us to adjust based on their changes in every case, which I
guess explains why 8.4.X has not been fixed either.

---------------------------------------------------------------------------

Martin von Gagern wrote:
-- Start of PGP signed section.
> Dear maintainers of PostgreSQL,
>
> Build errors for PostgreSQL have been reported on Gentoo Linux, see
> https://bugs.gentoo.org/285953 for the full bug report there.
>
> The issue has originally been reported for PostgreSQL 8.4.1, but has
> been reproduced with 8.4.2, 8.4.3 and even some 9.0 alpha.
>
> In short, the build fails when the heimdal kerberos implementation is
> installed on the system, and heimdal wasn't built with -Wl,--as-needed.
>
> In that case, the postgresql configure script detects that the strlcat
> and strlcpy functions as available. It does so because libgssapi.so and
> libkrb5.so both depend on heimdal's libroken.so, which currently does
> export these symbols. A fix for heimdal is in progress, which will
> reduce the list of exported symbols using a linker script, but afaik it
> hasn't been released yet, so for now heimdal installations providing
> these symbols are a fact of life.
>
> Later on, when postgresql has been configured and gets compiled, the
> -Wl,--as-needed flag is passed to the linker, and as libroken itself
> isn't named as a library to link against, the linker will complain about
> undefined references.
>
> Parts of the configure output:
> > checking whether strlcat is declared... no
> > checking whether strlcpy is declared... no
> > checking for strlcat... yes
> > checking for strlcpy... yes
>
> Parts of the make output:
> > descriptor.o: In function `descriptor_variable':
> > .../postgresql-8.4.1/src/interfaces/ecpg/preproc/descriptor.c:326: undefined reference to `strlcpy'
> > ../../../../src/port/libpgport.a(path.o): In function `get_home_path':
> > .../postgresql-8.4.1/src/port/path.c:644: undefined reference to `strlcpy'
> > ../../../../src/port/libpgport.a(path.o): In function `join_path_components':
> > .../postgresql-8.4.1/src/port/path.c:177: undefined reference to `strlcpy'
> > ../../../../src/port/libpgport.a(path.o): In function `make_relative_path':
> > .../postgresql-8.4.1/src/port/path.c:503: undefined reference to `strlcpy'
> > .../postgresql-8.4.1/src/port/path.c:523: undefined reference to `strlcpy'
> > collect2: ld returned 1 exit status
> > make[4]: *** [ecpg] Error 1
>
> Several solutions are conceivable. The Gentoo user Honza Mach??ek
> proposed handling libroken the same way you already handle libedit, i.e.
> disable some libraries during detection of these functions. See
> https://bugs.gentoo.org/attachment.cgi?id=214023 for his patch. In short
> it's this:
> > -# Some versions of libedit contain strlcpy(); so disregard that library while
> > +# Some versions of libedit and libroken contain strlcpy(); so disregard that library while
> >  # checking for these standard libc functions.
> >  pgac_save_LIBS="$LIBS"
> > -LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
> > +LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g' -e 's/-lgssapi//g' -e 's/-lkrb5//g'`
>
> Another alternative might be choosing behaviour based on the detection
> of suitable function declarations, i.e. HAVE_DECL_STRLCAT instead of
> HAVE_STRLCAT. Or you might try enabling the -Wl,--as-needed flag before
> detecting these library functions, to ensure the same behaviour at
> configure time and at build time. In both cases you might whish to
> rename those symbols in case you use your own implementation, just to
> make sure the linker knows which one to choose.
>
> Please choose a suitable method to fix this build error for PostgreSQL.
>
> Thanks,
>  Martin von Gagern
>
-- End of PGP section, PGP failed!

--
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Dear Bruce,

On 15.04.2010 03:59, Bruce Momjian wrote:
> I haven't seen anyone reply to this report,

Probably because Craig Ringer forwarded it to the -hackers list with cc
to me personally, but not to these lists. For that thread see
http://thread.gmane.org/gmane.comp.db.postgresql.devel.general/139424

> which might mean we are
> waiting for Gentoo to be fixed.  Gentoo is notoriously dynamic and it is
> hard for us to adjust based on their changes in every case, which I
> guess explains why 8.4.X has not been fixed either.

As I see it, Gentoo is one of the best ways to expose bugs in the build
system of any package, due to the high number of from-source builds
using various setups that Gentoo users perform. So I'd not say you'd be
adjusting pgsql to Gentoo, but you'd be fixing a bug in your build
system which Gentoo users have found and analyzed, but which might as
well hit other people building from source on their own.

So yes, Gentoo could address this at the distribution level, and as I
wrote on the Gentoo bug report, I hope they will do so soon. But the bug
is in the pgsql build system (and a bit in heimdal), so if you were to
apply one of the proposed fixes upstream, at least for future releases,
then more users could benefit.

> Martin von Gagern wrote:
>> Another alternative might be choosing behaviour based on the detection
>> of suitable function declarations, i.e. HAVE_DECL_STRLCAT instead of
>> HAVE_STRLCAT. Or you might try enabling the -Wl,--as-needed flag before
>> detecting these library functions, to ensure the same behaviour at
>> configure time and at build time. In both cases you might whish to
>> rename those symbols in case you use your own implementation, just to
>> make sure the linker knows which one to choose.

I've thought of one more alternative: Move the detection of gssapi and
krb5 libraries to a place after strcat/strcpy have been detected.

Greetings,
 Martin
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.14 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkvGtE0ACgkQRhp6o4m9dFvLqwCgmszpmy54y7oTxxNT6ssmTps+
J5IAoI9ENyPBVtZVsgSOfGPA1I3s/Wbl
=jpbU
-----END PGP SIGNATURE-----

Martin von Gagern <Martin.vGagern@gmx.net> writes:
> As I see it, Gentoo is one of the best ways to expose bugs in the build
> system of any package, due to the high number of from-source builds
> using various setups that Gentoo users perform. So I'd not say you'd be
> adjusting pgsql to Gentoo, but you'd be fixing a bug in your build
> system which Gentoo users have found and analyzed, but which might as
> well hit other people building from source on their own.

It's not a bug in our build system; it's a bug in Gentoo's library set,
ie, the heimdal library is exposing a symbol that it can't actually
guarantee to provide.  If it were a generic issue we'd be seeing it
reported on other Linuxen, since AFAIK nobody's glibc provides strlcpy.

The question for us is whether we should install a workaround, which in
any case would only be needed until Gentoo fixes their heimdal build.
I'm inclined to agree with Bruce that it's not worth the trouble ---
especially since I don't care for any of the solutions you proposed.
Most of them could break other platforms.

            regards, tom lane

Dear Tom.

On 16.04.2010 23:03, Tom Lane wrote:
> It's not a bug in our build system; it's a bug in Gentoo's library set,
> ie, the heimdal library is exposing a symbol that it can't actually
> guarantee to provide.

I still maintain that it's a bug in the build system: heimdal does
expose and provide the symbols via transitive dependencies. The
configure script tests for the symbol without explicitely enabling
--as-needed, and therefore uses the transitive dependency and finds the
symbol. The actual linker invocation later on during the build uses
--as-needed, therefore disregards the transitive dependencies, therefore
fails to find the symbol and errors out.

The idea of configure is to test whether features are actually usable
for a build. Therefore using different linker flags than at application
link time is against the idea of configure scripts.

I agree that it's not particularly nice of libraries like libroken to
export strlcpy and strlcat, as these are not part of its official job
description.

> If it were a generic issue we'd be seeing it
> reported on other Linuxen, since AFAIK nobody's glibc provides strlcpy.

Most binary distros use the MIT Kerberos implementation, at least when
building their packages, so they won't encounter this.
Try installing both heimdal and pgsql from source, without explicitely
using --as-needed for either, and if that fails to reproduce the issue,
I'll accept your statement.

> The question for us is whether we should install a workaround, which in
> any case would only be needed until Gentoo fixes their heimdal build.
> I'm inclined to agree with Bruce that it's not worth the trouble ---
> especially since I don't care for any of the solutions you proposed.
> Most of them could break other platforms.

How so? How is stripping kerberos libs for the function detection part
any worse than stripping libedit?

How could adding -Wl,--as-needed before the checks instead of after
break complete builds? Anywhere where it breaks detection, the build is
bound to break in a way similar to this issue here, isn't it?

If you can outline how this might break other platforms, maybe I or some
other Gentoo user can come up with an improved solution.

Greetings,
 Martin von Gagern