Обсуждение: Failure during initdb - creating dictionaries ... FATAL: could not access file "$libdir/libdict_snowball": No such file or directory

Hopefully this is a quick one to answer, but it's got me tearing my
hair out.

Custom built postgresql 8.3.5 using the pkgsrc build system on OS X
Leopard; when running: /usr/pkg/bin/initdb -D /usr/pkg/pgsql/data
(I also tried adding the -d flag but no other useful information was
shown)
as the pgsql user, it seems to be dying with a dictionary problem as
follows:

sh-3.2$ /usr/pkg/bin/initdb -D /usr/pkg/pgsql/data
The files belonging to this database system will be owned by user
"pgsql".
This user must also own the server process.

The database cluster will be initialized with locale en_US.UTF-8.
The default database encoding has accordingly been set to UTF8.
The default text search configuration will be set to "english".

fixing permissions on existing directory /usr/pkg/pgsql/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers/max_fsm_pages ... 12MB/76800
creating configuration files ... ok
creating template1 database in /usr/pkg/pgsql/data/base/1 ... ok
initializing pg_authid ... ok
initializing dependencies ... ok
creating system views ... ok
loading system objects' descriptions ... ok
creating conversions ... ok
creating dictionaries ... FATAL:  could not access file
"$libdir/libdict_snowball": No such file or directory
STATEMENT:  -- Language-specific snowball dictionaries
        -- $PostgreSQL:
        pgsql/src/backend/snowball/snowball_func.sql.in,v 1.2
        2007/09/03 02:30:43 tgl Exp $$

        SET search_path = pg_catalog;

        CREATE FUNCTION dsnowball_init(INTERNAL)
            RETURNS INTERNAL AS '$libdir/libdict_snowball',
        'dsnowball_init'
        LANGUAGE C STRICT;
....(snip)


......many  many lines of snowball specific SQL and then finally....
    ALTER TEXT SEARCH CONFIGURATION turkish ADD MAPPING
        FOR asciiword, hword_asciipart, asciihword
               WITH turkish_stem;

       ALTER TEXT SEARCH CONFIGURATION turkish ADD MAPPING
           FOR word, hword_part, hword
                  WITH turkish_stem;


child process exited with exit code 1
initdb: removing contents of data directory "/usr/pkg/pgsql/data"

===============
So, presumably the fatal error which resulted in exit code 1 was:
FATAL:  could not access file "$libdir/libdict_snowball": No such file

If so, how to fix so that initdb can complete?

Further details:
pg_config --libdir
/Volumes/pkgsrc/pkg/lib

pg_config --pkglibdir
/Volumes/pkgsrc/pkg/lib/postgresql

sh-3.2$ ls -l /Volumes/pkgsrc/pkg/lib/postgresql | grep snowball
-rwxr-xr-x  1 root  wheel   341480 Feb 27 17:06
libdict_snowball.0.0.0.dylib
lrwxr-xr-x  1 root  wheel       28 Feb 27 17:06
libdict_snowball.0.dylib -> libdict_snowball.0.0.0.dylib
lrwxr-xr-x  1 root  wheel       28 Feb 27 17:06 libdict_snowball.dylib
-> libdict_snowball.0.0.0.dylib
-rwxr-xr-x  1 root  wheel      893 Feb 27 17:06 libdict_snowball.la

And I've symlinked these all from /Volumes/pkgsrc/pkg/lib as well, but
initdb still fails to complete.

Any help will be gratefully apreciated.

Regards,
Jonathan.

Jonathan Groll <lists@groll.co.za> writes:
> Custom built postgresql 8.3.5 using the pkgsrc build system on OS X
> Leopard;

Uh ... what is the "pkgsrc build system", and what changes does it make
to a straight-from-source PG build?

> creating conversions ... ok
> creating dictionaries ... FATAL:  could not access file
> "$libdir/libdict_snowball": No such file or directory

What you've apparently got here is a problem with shared-library
building, but it's odd that it would've got through the "creating
conversions" step, which also relies heavily on shared libraries.

> sh-3.2$ ls -l /Volumes/pkgsrc/pkg/lib/postgresql | grep snowball
> -rwxr-xr-x  1 root  wheel   341480 Feb 27 17:06
> libdict_snowball.0.0.0.dylib
> lrwxr-xr-x  1 root  wheel       28 Feb 27 17:06
> libdict_snowball.0.dylib -> libdict_snowball.0.0.0.dylib
> lrwxr-xr-x  1 root  wheel       28 Feb 27 17:06 libdict_snowball.dylib
> -> libdict_snowball.0.0.0.dylib
> -rwxr-xr-x  1 root  wheel      893 Feb 27 17:06 libdict_snowball.la

I find this pretty suspicious.  On my OSX machine, the contents of the
pkglibdir directory are all named something.so:

$ ls version83/lib/postgresql
ascii_and_mic.so*                       utf8_and_euc_cn.so*
cyrillic_and_mic.so*                    utf8_and_euc_jis_2004.so*
dict_snowball.so*                       utf8_and_euc_jp.so*
euc_cn_and_mic.so*                      utf8_and_euc_kr.so*
euc_jis_2004_and_shift_jis_2004.so*     utf8_and_euc_tw.so*
euc_jp_and_sjis.so*                     utf8_and_gb18030.so*
euc_kr_and_mic.so*                      utf8_and_gbk.so*
euc_tw_and_big5.so*                     utf8_and_iso8859.so*
latin2_and_win1250.so*                  utf8_and_iso8859_1.so*
latin_and_mic.so*                       utf8_and_johab.so*
pgxs/                                   utf8_and_shift_jis_2004.so*
plpgsql.so*                             utf8_and_sjis.so*
utf8_and_ascii.so*                      utf8_and_uhc.so*
utf8_and_big5.so*                       utf8_and_win.so*
utf8_and_cyrillic.so*
$

Notice also the lack of the "lib" prefix.  The source code that you are
quoting should look like this:

CREATE FUNCTION dsnowball_init(INTERNAL)
    RETURNS INTERNAL AS '$libdir/dict_snowball', 'dsnowball_init'
LANGUAGE C STRICT;

so how did it get to be 'libdir/libdict_snowball'?

It looks to me like you are using code that someone has hacked up to try
to impose their own ideas of how to build/name shared libraries for OS X,
and done a pretty darn poor job of it.

Recommendation: get an unmodified copy of Postgres and build it
yourself.  It works fine out-of-the-box on OS X.

            regards, tom lane

On Wed, May 13, 2009 at 09:54:56AM -0400, Tom Lane wrote:
>Jonathan Groll <lists@groll.co.za> writes:
>> Custom built postgresql 8.3.5 using the pkgsrc build system on OS X
>> Leopard;
>
>Uh ... what is the "pkgsrc build system", and what changes does it make
>to a straight-from-source PG build?

Pkgsrc is the netbsd package management system(0), used by some folks
(like myself) on OS X to build all of our open source packages in the
same way as others use macports or fink. It's normally very good.

>
>> creating conversions ... ok
>> creating dictionaries ... FATAL:  could not access file
>> "$libdir/libdict_snowball": No such file or directory
>
>What you've apparently got here is a problem with shared-library
>building, but it's odd that it would've got through the "creating
>conversions" step, which also relies heavily on shared libraries.

Agreed... I'm going to drop a note on the pkgsrc lists since it is now
clear the problem lies there.

>I find this pretty suspicious.  On my OSX machine, the contents of the
>pkglibdir directory are all named something.so:
>dict_snowball.so*                       utf8_and_euc_jp.so*

And mine are also .so after doing a regular make build with the same
tarball (with the exception of those whose name starts with lib*). I
was under the impression that OS X used .dylib for shared libraries(1)
- look in /usr/lib for example.

>Recommendation: get an unmodified copy of Postgres and build it
>yourself.  It works fine out-of-the-box on OS X.
>

Thank you for that it worked like a charm. I am also somewhat relieved
that the Makefile allows for 'make uninstall' so that some degree of
"package management" is possible.

Kind regards,
Jonathan Groll.

(0) http://netbsd.org/docs/software/packages.html
(1) http://osxbook.com/book/bonus/ancient/whatismacosx/programming.html

Jonathan Groll <lists@groll.co.za> writes:
> On Wed, May 13, 2009 at 09:54:56AM -0400, Tom Lane wrote:
>> I find this pretty suspicious.  On my OSX machine, the contents of the
>> pkglibdir directory are all named something.so:
>> dict_snowball.so*                       utf8_and_euc_jp.so*

> And mine are also .so after doing a regular make build with the same
> tarball (with the exception of those whose name starts with lib*). I
> was under the impression that OS X used .dylib for shared libraries(1)
> - look in /usr/lib for example.

I don't recall the details at the moment, but I think we intentionally
didn't adopt the .dylib extension for these files because of some subtle
difference between them and plain shared libraries.  I notice that
"file" describes them as "bundles" whereas what's in /usr/lib seems to
get described as "dynamically linked shared library" ... but the
implications escape my memory.

            regards, tom lane

Tom Lane wrote:

> I don't recall the details at the moment, but I think we intentionally
> didn't adopt the .dylib extension for these files because of some subtle
> difference between them and plain shared libraries.

If I recall correctly from porting a plugin-based app to Mac OS X a few
years ago, there are issues with dlopen(...) on .dylib libraries.

I seem to remember having issues getting dylibs to resolve symbols from
the loading executable and already-loaded libraries; they wanted to
resolve all symbols as direct dependencies. I needed to use a .so to get
the library to resolve symbols in the loading application.

A quick bit of reading shows that:

  - The extension doesn't matter, the OS doesn't care

  - The library type, MH_DYLIB or MH_BUNDLE, is controlled by how
    it's built only

  - MH_DYLIB won't resolve symbols from the loading executable
    and can't be unloaded, whereas MH_BUNDLE does and can.

  - Most libraries on Mac OS X are MH_DYLIB

Found some info at:


http://developer.apple.com/documentation/DeveloperTools/Conceptual/MachORuntime/Reference/reference.html#//apple_ref/doc/uid/20001298-BAJHHFFF

   under Fields, section "filetype"

--
Craig Ringer

makes you wonder what possible use a MH_DYLIB library might serve

Martin
______________________________________________
Disclaimer and Confidentiality/Verzicht und Vertraulichkeitanmerkung/Note de déni et de confidentialité

This message is confidential. If you should not be the intended receiver, then we ask politely to report. Each unauthorized forwarding or manufacturing of a copy is inadmissible. This message serves only for the exchange of information and has no legal binding effect. Due to the easy manipulation of emails we cannot take responsibility over the the contents.
Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene Empfaenger sein, so bitten wir hoeflich um eine Mitteilung. Jede unbefugte Weiterleitung oder Fertigung einer Kopie ist unzulaessig. Diese Nachricht dient lediglich dem Austausch von Informationen und entfaltet keine rechtliche Bindungswirkung. Aufgrund der leichten Manipulierbarkeit von E-Mails koennen wir keine Haftung fuer den Inhalt uebernehmen.
Ce message est confidentiel et peut être privilégié. Si vous n'êtes pas le destinataire prévu, nous te demandons avec bonté que pour satisfaire informez l'expéditeur. N'importe quelle diffusion non autorisée ou la copie de ceci est interdite. Ce message sert à l'information seulement et n'aura pas n'importe quel effet légalement obligatoire. Étant donné que les email peuvent facilement être sujets à la manipulation, nous ne pouvons accepter aucune responsabilité pour le contenu fourni.

> Date: Fri, 15 May 2009 08:36:48 +0800
> From: craig@postnewspapers.com.au
> To: tgl@sss.pgh.pa.us
> CC: lists@groll.co.za; pgsql-general@postgresql.org
> Subject: Re: [GENERAL] Failure during initdb - creating dictionaries ... FATAL: could not access file "$libdir/libdict_snowball": No such file or directory
>
> Tom Lane wrote:
>
> > I don't recall the details at the moment, but I think we intentionally
> > didn't adopt the .dylib extension for these files because of some subtle
> > difference between them and plain shared libraries.
>
> If I recall correctly from porting a plugin-based app to Mac OS X a few
> years ago, there are issues with dlopen(...) on .dylib libraries.
>
> I seem to remember having issues getting dylibs to resolve symbols from
> the loading executable and already-loaded libraries; they wanted to
> resolve all symbols as direct dependencies. I needed to use a .so to get
> the library to resolve symbols in the loading application.
>
> A quick bit of reading shows that:
>
> - The extension doesn't matter, the OS doesn't care
>
> - The library type, MH_DYLIB or MH_BUNDLE, is controlled by how
> it's built only
>
> - MH_DYLIB won't resolve symbols from the loading executable
> and can't be unloaded, whereas MH_BUNDLE does and can.
>
> - Most libraries on Mac OS X are MH_DYLIB
>
> Found some info at:
>
> http://developer.apple.com/documentation/DeveloperTools/Conceptual/MachORuntime/Reference/reference.html#//apple_ref/doc/uid/20001298-BAJHHFFF
>
> under Fields, section "filetype"
>
> --
> Craig Ringer
>
> --
> Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-general

---

Hotmail® has a new way to see what's up with your friends. Check it out.