Обсуждение: BUG #4186: set lc_messages does not work
The following bug has been logged online: Bug reference: 4186 Logged by: Thomas H Email address: me@alternize.com PostgreSQL version: 8.3.1 Operating system: Windows 2003 Description: set lc_messages does not work Details: the patch discussed here [1] that supposedly made the win32 msvc builds use lc_locale properly has flaws. while indeed it does force pgsql to use the native win32 locale for error messages (before 8.3.1, system messages were always english), it broke the functionality to actually have the lc_messages locale set manually through conf or in a user session. the following output is from a win2003 system with German_Switzerland.1252 locale. the queries are identically to the one used as examples by the patch author. i have added a "select x;" to trigger a system error message to show that its actually misbehaving - all output is always in (swiss) german despite the set locale. - thomas [1] http://archives.postgresql.org/pgsql-patches/2008-02/msg00038.php -------------------------------- Dies ist psql 8.3.1, das interaktive PostgreSQL-Terminal. Geben Sie ein: \copyright für Urheberrechtsinformationen \h für Hilfe über SQL-Anweisungen \? für Hilfe über interne Anweisungen \g oder Semikolon, um eine Anfrage auszuführen \q um zu beenden Warnung: Konsolencodeseite (850) unterscheidet sich von der Windows- Codeseite (1252). 8-Bit-Zeichen funktionieren möglicherweise nicht richtig. Einzelheiten finden Sie auf der psql-Handbuchseite unter »Notes for Windows users«. endor=# set lc_messages to 'sv_SE'; SET endor=# select to_char((current_date + s.a),'TMDay TMMonth YYYY') as dates from generate_series(0,6) as s(a); dates --------------------- Donnerstag Mai 2008 Freitag Mai 2008 Samstag Mai 2008 Sonntag Mai 2008 Montag Mai 2008 Dienstag Mai 2008 Mittwoch Mai 2008 (7 Zeilen) endor=# select x; FEHLER: Spalte »x« existiert nicht ZEILE 1: select x; ^ endor=# endor=# set lc_messages to 'de_DE'; SET endor=# select to_char((current_date + s.a),'TMDay TMMonth YYYY') as dates from generate_series(0,6) as s(a); dates --------------------- Donnerstag Mai 2008 Freitag Mai 2008 Samstag Mai 2008 Sonntag Mai 2008 Montag Mai 2008 Dienstag Mai 2008 Mittwoch Mai 2008 (7 Zeilen) endor=# select x; FEHLER: Spalte »x« existiert nicht ZEILE 1: select x; ^ endor=# endor=# set lc_messages to 'English_United_States'; SET endor=# select to_char((current_date + s.a),'TMDay TMMonth YYYY') as dates from generate_series(0,6) as s(a); dates --------------------- Donnerstag Mai 2008 Freitag Mai 2008 Samstag Mai 2008 Sonntag Mai 2008 Montag Mai 2008 Dienstag Mai 2008 Mittwoch Mai 2008 (7 Zeilen) endor=# select x; FEHLER: Spalte »x« existiert nicht ZEILE 1: select x; ^ endor=# endor=# set lc_messages to 'fr'; SET endor=# select to_char((current_date + s.a),'TMDay TMMonth YYYY') as dates from generate_series(0,6) as s(a); dates --------------------- Donnerstag Mai 2008 Freitag Mai 2008 Samstag Mai 2008 Sonntag Mai 2008 Montag Mai 2008 Dienstag Mai 2008 Mittwoch Mai 2008 (7 Zeilen) endor=# select x; FEHLER: Spalte »x« existiert nicht ZEILE 1: select x;
Thomas H wrote: > the patch discussed here [1] that supposedly made the win32 msvc builds use > lc_locale properly has flaws. > I think you misunderstood the feature [1] added recently. This new feature doesn't rely on lc_messages to localize the day and month names; it uses lc_time. Look at the archives for reference. [1] http://archives.postgresql.org/pgsql-patches/2008-04/msg00451.php -- Euler Taveira de Oliveira http://www.timbira.com/
euler taveira de olivieira wrote:
>> the patch discussed here [1] that supposedly made the win32 msvc
>> builds use
>> lc_locale properly has flaws.
>>
> I think you misunderstood the feature [1] added recently. This new
actually no. the problem is as i intended to point out with the system
generated error messages. their language is supposedly controlled by
LC_MESSAGES (as was the time/date output, according to the first
february patch submission).
please observe the (previously already submitted) test queries. i've
removed the date/time testqueries to no further distract from the
problem. the bogus query "select x;" always results in a german error
messages no matter what LC_MESSAGES is set:
endor=# set lc_messages to 'sv_SE';
SET
endor=# select x;
FEHLER: Spalte »x« existiert nicht
ZEILE 1: select x;
^
endor=#
endor=# set lc_messages to 'de_DE';
SET
endor=# select x;
FEHLER: Spalte »x« existiert nicht
ZEILE 1: select x;
^
endor=#
endor=# set lc_messages to 'English_United_States';
SET
endor=# select x;
FEHLER: Spalte »x« existiert nicht
ZEILE 1: select x;
^
endor=#
endor=# set lc_messages to 'fr';
SET
endor=# select x;
FEHLER: Spalte »x« existiert nicht
ZEILE 1: select x;
setting LC_MESSAGES in postgres.conf doesn't change anything either. and
the comment there says explicitly that LC_MESSAGES is used for system
errors:
lc_messages = 'English_United_States' # locale for system error message
# strings
nevertheless, everything is outputted in german now, errors as well as
logs. which seems to hint at 8.3.1 ignoring the LC_MESSAGES and always
using the os' locale.
regards,
thomas
Thomas H. wrote: > please observe the (previously already submitted) test queries. i've > removed the date/time testqueries to no further distract from the > problem. the bogus query "select x;" always results in a german error > messages no matter what LC_MESSAGES is set: > OK, that's another problem. AFAIK, that's a known problem because Windows doesn't have LC_MESSAGES. The above comment (pg_locale.c) suggests that there is no verification for the locale that is been set. A possible solution is to use IsValidLocaleName() [1] or LocaleNameToLCID() [2] but it seems that they're only available for Vista. :( Maybe we could emulate one of these functions with a mapping table [3]. [searching ...] It seems there are problems with LCIDs; they don't describe the locales acurately. pgwin hackers? Could you try to use one of the locale names described in [4]? /* * LC_MESSAGES category does not exist everywhere, but accept it anyway * * On Windows, we can't even check the value, so the non-doit case is a * no-op */ [1] http://msdn.microsoft.com/en-us/library/ms776379(VS.85).aspx [2] http://msdn.microsoft.com/en-us/library/ms776388(VS.85).aspx [3] http://msdn.microsoft.com/en-us/library/ms776260.aspx -- Euler Taveira de Oliveira http://www.timbira.com/
Euler Taveira de Oliveira wrote:
>> please observe the (previously already submitted) test queries. i've
>> removed the date/time testqueries to no further distract from the
>> problem. the bogus query "select x;" always results in a german error
>> messages no matter what LC_MESSAGES is set:
>>
> OK, that's another problem. AFAIK, that's a known problem because
> Windows doesn't have LC_MESSAGES. The above comment (pg_locale.c)
> suggests that there is no verification for the locale that is been set.
> A possible solution is to use IsValidLocaleName() [1] or
> LocaleNameToLCID() [2] but it seems that they're only available for
> Vista. :( Maybe we could emulate one of these functions with a mapping
> table [3]. [searching ...] It seems there are problems with LCIDs; they
> don't describe the locales acurately. pgwin hackers?
how does LC_MESSAGES differ from for example LC_TIME? in LC_TIME, the
checking of the specified locale seems to work:
endor=# set LC_MESSAGES = 'en-US';
SET
endor=# select x;
FEHLER: Spalte »x« existiert nicht
ZEILE 1: select x;
^
endor=# set LC_TIME = 'en-US';
FEHLER: ungültiger Wert für Parameter »lc_time«: »en-US«
endor=# set LC_TIME = 'en';
FEHLER: ungültiger Wert für Parameter »lc_time«: »en«
endor=# set LC_TIME = 'English';
SET
maybe one could as a workaround just use the lc_time locale checks for
lc_messages on windows systems? or at least match against the internal
pgsql supported translations. i don't mind having to specify "en"
instead of "English" if that gets me english error messages ;)
> Could you try to use one of the locale names described in [4]?
i take it you meant link [3]. i've tried 'en-US' and others, same
problem, errors in german (excerpt above).
> [1] http://msdn.microsoft.com/en-us/library/ms776379(VS.85).aspx
> [2] http://msdn.microsoft.com/en-us/library/ms776388(VS.85).aspx
> [3] http://msdn.microsoft.com/en-us/library/ms776260.aspx
>
regards,
thomas
"Thomas H" <me@alternize.com> writes:
> Bug reference: 4186
> Logged by: Thomas H
> Email address: me@alternize.com
> PostgreSQL version: 8.3.1
> Operating system: Windows 2003
> Description: set lc_messages does not work
> Details:
> the patch discussed here [1] that supposedly made the win32 msvc builds use
> lc_locale properly has flaws.
I think a large part of the confusion that's been evidenced in this
thread is because you are submitting a "bug report" about a patch that is
not in fact in 8.3.1 (and is unlikely ever to appear in 8.3.x at all).
It's unclear what you are really testing: stock 8.3.1? 8.3.1 plus that
patch? CVS HEAD plus the patch?
regards, tom lane
From: Tom Lane <tgl@sss.pgh.pa.us>
>> the patch discussed here [1] that supposedly made the win32 msvc builds use
>> lc_locale properly has flaws.
>
> I think a large part of the confusion that's been evidenced in this
> thread is because you are submitting a "bug report" about a patch that is
> not in fact in 8.3.1 (and is unlikely ever to appear in 8.3.x at all).
> It's unclear what you are really testing: stock 8.3.1? 8.3.1 plus that
> patch? CVS HEAD plus the patch?
it's the 8.3.1 win32 binary coming from postgresql.org i'm using.
i was digging in the mailing list archives for some answers and found
the conversation about the patch which i believed to be included in
8.3.1. sorry for the mess :|
nevertheless the problem/bug exists: changing LC_MESSAGES has no effect
on the windows boxes, while it works on the non-win32 systems. all i
really would like is to get english system messages back on our
non-english win32 servers - like they were pre 8.3.1.
the main reason for this being: the german error messages have non-ascii
special characters in the error text ('»' and '«') which mess with some
of our applications (whereas english messages use '"').
regards,
thomas
"Thomas H." <me@alternize.com> writes:
> nevertheless the problem/bug exists: changing LC_MESSAGES has no effect
> on the windows boxes, while it works on the non-win32 systems. all i
> really would like is to get english system messages back on our
> non-english win32 servers - like they were pre 8.3.1.
So far as I can tell, the backend's handling of LC_MESSAGES hasn't
changed at all between 8.1.2 and 8.3.1, so if it used to work for you
then there's been some other relevant change. Any idea what?
regards, tom lane
From: Tom Lane <tgl@sss.pgh.pa.us>
> "Thomas H." <me@alternize.com> writes:
>> nevertheless the problem/bug exists: changing LC_MESSAGES has no effect
>> on the windows boxes, while it works on the non-win32 systems. all i
>> really would like is to get english system messages back on our
>> non-english win32 servers - like they were pre 8.3.1.
>
> So far as I can tell, the backend's handling of LC_MESSAGES hasn't
> changed at all between 8.1.2 and 8.3.1, so if it used to work for you
> then there's been some other relevant change. Any idea what?
well... i'm not saying it worked in earlier versions. but prior to 8.3.1
(i've tested 8.3.0 and 8.2.3), the error messages were in english no
matter what the config file or the os locale says.
now, with 8.3.1, error & system messages are always in the os' locale,
and thus the bugreport. there is currently no way to set the pg system
messages' language, as the LC_MESSAGES setting seems to be defunct on
win32 systems.
as a small "proof", i installed the win32 8.3.0 from postgresql.org as a
fresh install and changed the postgresl.conf' lc_messages value to
'English_UnitedStates'. here's the psql output:
8.3.0:
-----------------------------------
postgres=# show lc_messages;
lc_messages
--------------------
English_UnitedStates
(1 row)
postgres=# select x;
ERROR: column "x" does not exist
LINE 1: select x;
^
postgres=# set lc_messages='French';
SET
postgres=# select x;
ERROR: column "x" does not exist
LINE 1: select x;
^
postgres=#
-----------------------------------
--> all system messages are in english.
after *upgrading* to 8.3.1 (again, using the official binaries), the
output looks like this:
8.3.1:
-----------------------------------
postgres=# show lc_messages;
lc_messages
----------------------
English_UnitedStates
(1 Zeile)
postgres=# select x;
FEHLER: Spalte »x« existiert nicht
ZEILE 1: select x;
^
postgres=# set lc_messages='French';
SET
postgres=# select x;
FEHLER: Spalte »x« existiert nicht
ZEILE 1: select x;
^
postgres=# show lc_messages;
lc_messages
-------------
French
(1 Zeile)
postgres=#
-----------------------------------
--> all system messages are in german (the os' locale)
so clearly between 8.3.0 and 8.3.1, something must have changed. but the
only patch that concerned win32 msvc/locale is the one you said wasn't
even included...
what i noticed: if i delete the folder share/locale/de/ the system
messages are back to english - but that can't be THE solution, can it? :)
regards,
thomas
From: Thomas H. <me@alternize.com> > what i noticed: if i delete the folder share/locale/de/ the system > messages are back to english - but that can't be THE solution, can it? :) well, it actually was the solution, at least to the weird part of the problem: there are two versions of win32 binaries available on postgres.org. there's the installer msi version, and there is the installerless zip version. the installer version does NOT install the folder "share/locale/"! so when using the msi installer, postgres has no translations at all - thus the fallback to english. the zipped version contains the "share/locale/" folder. for installing 8.3.0, i've been using the msi installer version. for upgrading to 8.3.1, i've been unpacking the files from zip version into the postgresql directory... i've just verified that the 8.3.1 msi version provided on postgres.org also does NOT contain the "locale" folder & files. should i report this as a separate bug/problem? so at least that explains the "changed" behaviour. nevertheless, LC_MESSAGES seems to be defunct - with the "locale" folder present, pg always picks the os' language and ignores the lc_message value. - thomas
On Thu, May 29, 2008 at 2:05 AM, Thomas H. <me@alternize.com> wrote: > From: Thomas H. <me@alternize.com> >> > i've just verified that the 8.3.1 msi version provided on postgres.org also > does NOT contain the "locale" folder & files. should i report this as a > separate bug/problem? How exactly did you do that? My installation certainly has it. If memory serves, it's not installed by default (you have to select it in the feature list), but it's there alright. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com
From: Dave Page <dpage@pgadmin.org> > On Thu, May 29, 2008 at 2:05 AM, Thomas H. <me@alternize.com> wrote: >> From: Thomas H. <me@alternize.com> >> i've just verified that the 8.3.1 msi version provided on postgres.org also >> does NOT contain the "locale" folder & files. should i report this as a >> separate bug/problem? > > How exactly did you do that? My installation certainly has it. If > memory serves, it's not installed by default (you have to select it in > the feature list), but it's there alright. > hmm by just clicking through the standard settings. i've seen now that the "national language support" is set to do not install by default. so it is a feature, not a bug, sorry. i was under the obviously wrong impression a zip-file upgrade would be the same as an msi-upgrade. - thomas
On Thu, May 29, 2008 at 12:23 PM, Thomas H. <me@alternize.com> wrote: > i was under the obviously wrong impression a zip-file upgrade would be the > same as an msi-upgrade. Eeep - don't do that!! That is a *very* good way to annoy the Windows Installer and trick it into thinking things need to be repaired. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com
Thomas H. wrote: > From: Thomas H. <me@alternize.com> > > what i noticed: if i delete the folder share/locale/de/ the system > > messages are back to english - but that can't be THE solution, can > > it? :) > > well, it actually was the solution, at least to the weird part of the > problem: > > there are two versions of win32 binaries available on postgres.org. > there's the installer msi version, and there is the installerless zip > version. > > the installer version does NOT install the folder "share/locale/"! so > when using the msi installer, postgres has no translations at all - > thus the fallback to english. Are you sure you didn't just skip enabling National Language Support? It's not included by default, but should be available. > so at least that explains the "changed" behaviour. nevertheless, > LC_MESSAGES seems to be defunct - with the "locale" folder present, > pg always picks the os' language and ignores the lc_message value. This looks like I can reproduce though, at least on cvs head. Did this work for you in previous versions? //Magnus
Magnus Hagander <magnus@hagander.net> writes: > Thomas H. wrote: >> so at least that explains the "changed" behaviour. nevertheless, >> LC_MESSAGES seems to be defunct - with the "locale" folder present, >> pg always picks the os' language and ignores the lc_message value. > This looks like I can reproduce though, at least on cvs head. Did this > work for you in previous versions? Maybe we were using a different build of gettext in the previous releases, one that didn't look at the same info as the current code? Anyway the patch mentioned at the start of the thread http://archives.postgresql.org/pgsql-patches/2008-02/msg00038.php purports to fix this. It doesn't seem to have gotten reviewed though. regards, tom lane
Tom Lane wrote: > Magnus Hagander <magnus@hagander.net> writes: > > Thomas H. wrote: > >> so at least that explains the "changed" behaviour. nevertheless, > >> LC_MESSAGES seems to be defunct - with the "locale" folder present, > >> pg always picks the os' language and ignores the lc_message value. > > > This looks like I can reproduce though, at least on cvs head. Did this > > work for you in previous versions? > > Maybe we were using a different build of gettext in the previous > releases, one that didn't look at the same info as the current code? > > Anyway the patch mentioned at the start of the thread > http://archives.postgresql.org/pgsql-patches/2008-02/msg00038.php > purports to fix this. It doesn't seem to have gotten reviewed > though. Agreed. Magnus, someone, can we get feedback on the patch at this URL? http://archives.postgresql.org/pgsql-patches/2008-02/msg00038.php -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian wrote: > Tom Lane wrote: >> Magnus Hagander <magnus@hagander.net> writes: >>> Thomas H. wrote: >>>> so at least that explains the "changed" behaviour. nevertheless, >>>> LC_MESSAGES seems to be defunct - with the "locale" folder present, >>>> pg always picks the os' language and ignores the lc_message value. >>> This looks like I can reproduce though, at least on cvs head. Did this >>> work for you in previous versions? >> Maybe we were using a different build of gettext in the previous >> releases, one that didn't look at the same info as the current code? >> >> Anyway the patch mentioned at the start of the thread >> http://archives.postgresql.org/pgsql-patches/2008-02/msg00038.php >> purports to fix this. It doesn't seem to have gotten reviewed >> though. > > Agreed. Magnus, someone, can we get feedback on the patch at this URL? > > http://archives.postgresql.org/pgsql-patches/2008-02/msg00038.php IIRC, there was further work to be done on the patch before it was to be applied, and we held off the review until then. Gevik - can you comment on this? Where are we, what needs ot be done still? //Magnus
Hi Magnus. I have tried with Inoue-san, investigation of this problem, and adjustment. http://winpg.jp/~saito/pg_work/LC_MESSAGE_CHECK/LC_TIME_PATCH/pg8.3.3-to_char_gettext_format.png Native-strftime was proposed by Tom-san. It corrects(LC_TIME) from 8.3.3.(LC_MESSAGES) http://winpg.jp/~saito/pg_work/LC_MESSAGE_CHECK/LC_TIME_PATCH/ http://winpg.jp/~saito/pg_work/LC_MESSAGE_CHECK/LC_TIME_PATCH/pg_locale_patch It has not resulted in the still final conclusion. However, Possibly this may be the optimal one. http://winpg.jp/~saito/pg_work/LC_MESSAGE_CHECK/LC_TIME_PATCH/wcsftimeck.c I am creating the proposal of patch with Inoue-san. Regards, Hiroshi Saito ----- Original Message ----- From: "Magnus Hagander" <magnus@hagander.net> > Bruce Momjian wrote: >> Tom Lane wrote: >>> Magnus Hagander <magnus@hagander.net> writes: >>>> Thomas H. wrote: >>>>> so at least that explains the "changed" behaviour. nevertheless, >>>>> LC_MESSAGES seems to be defunct - with the "locale" folder present, >>>>> pg always picks the os' language and ignores the lc_message value. >>>> This looks like I can reproduce though, at least on cvs head. Did this >>>> work for you in previous versions? >>> Maybe we were using a different build of gettext in the previous >>> releases, one that didn't look at the same info as the current code? >>> >>> Anyway the patch mentioned at the start of the thread >>> http://archives.postgresql.org/pgsql-patches/2008-02/msg00038.php >>> purports to fix this. It doesn't seem to have gotten reviewed >>> though. >> >> Agreed. Magnus, someone, can we get feedback on the patch at this URL? >> >> http://archives.postgresql.org/pgsql-patches/2008-02/msg00038.php > > IIRC, there was further work to be done on the patch before it was to be > applied, and we held off the review until then. > > Gevik - can you comment on this? Where are we, what needs ot be done still? > > //Magnus > > > -- > Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-bugs
My apologies on this late reply. The way LC_MESSAGES is handled on windows is much less efficient and faulty. While ago I started with a patch to fix some of the issues I encountered on windows and LC_MESSAGES. But I stopped working on that patch because this problem needed to be fixed on many other places. In Windows, handling LC_MESSAGES will not work the same way as *nix systems, forcing us to make ugly workarounds. (as I actually was doing with my patch) To my opinion, unless we think of a coherent solution for handling LC_MESSAGES/locale for both *nix and win32 platforms, fixing lc_messages and locale issues would break more than fixing it. BTW: The gettext library we are using on win32 is a very old one. Regards, Gevik. > -----Original Message----- > From: Magnus Hagander [mailto:magnus@hagander.net] > Sent: Tuesday, August 05, 2008 4:54 PM > To: Bruce Momjian > Cc: Tom Lane; Thomas H.; pgsql-bugs@postgresql.org; Gevik Babakhani > Subject: Re: [BUGS] BUG #4186: set lc_messages does not work > > Bruce Momjian wrote: > > Tom Lane wrote: > >> Magnus Hagander <magnus@hagander.net> writes: > >>> Thomas H. wrote: > >>>> so at least that explains the "changed" behaviour. nevertheless, > >>>> LC_MESSAGES seems to be defunct - with the "locale" > folder present, > >>>> pg always picks the os' language and ignores the > lc_message value. > >>> This looks like I can reproduce though, at least on cvs head. Did > >>> this work for you in previous versions? > >> Maybe we were using a different build of gettext in the previous > >> releases, one that didn't look at the same info as the > current code? > >> > >> Anyway the patch mentioned at the start of the thread > >> http://archives.postgresql.org/pgsql-patches/2008-02/msg00038.php > >> purports to fix this. It doesn't seem to have gotten reviewed > >> though. > > > > Agreed. Magnus, someone, can we get feedback on the patch > at this URL? > > > > > http://archives.postgresql.org/pgsql-patches/2008-02/msg00038.php > > IIRC, there was further work to be done on the patch before > it was to be applied, and we held off the review until then. > > Gevik - can you comment on this? Where are we, what needs ot > be done still? > > //Magnus >
Tom Lane wrote: > Magnus Hagander <magnus@hagander.net> writes: > > Thomas H. wrote: > >> so at least that explains the "changed" behaviour. nevertheless, > >> LC_MESSAGES seems to be defunct - with the "locale" folder present, > >> pg always picks the os' language and ignores the lc_message value. > > > This looks like I can reproduce though, at least on cvs head. Did this > > work for you in previous versions? > > Maybe we were using a different build of gettext in the previous > releases, one that didn't look at the same info as the current code? > > Anyway the patch mentioned at the start of the thread > http://archives.postgresql.org/pgsql-patches/2008-02/msg00038.php > purports to fix this. It doesn't seem to have gotten reviewed > though. Where are we on this? -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Gevik Babakhani wrote: > My apologies on this late reply. > The way LC_MESSAGES is handled on windows is much less efficient and faulty. > While ago I started with a patch to fix some of the issues I encountered on > windows and LC_MESSAGES. But I stopped working on that patch because this > problem needed to be fixed on many other places. In Windows, handling > LC_MESSAGES will not work the same way as *nix systems, forcing us to make > ugly workarounds. (as I actually was doing with my patch) > > To my opinion, unless we think of a coherent solution for handling > LC_MESSAGES/locale for both *nix and win32 platforms, fixing lc_messages and > locale issues would break more than fixing it. > > BTW: The gettext library we are using on win32 is a very old one. This is the most recent posting on the topic, I think. Status? --------------------------------------------------------------------------- > > Regards, > Gevik. > > > > -----Original Message----- > > From: Magnus Hagander [mailto:magnus@hagander.net] > > Sent: Tuesday, August 05, 2008 4:54 PM > > To: Bruce Momjian > > Cc: Tom Lane; Thomas H.; pgsql-bugs@postgresql.org; Gevik Babakhani > > Subject: Re: [BUGS] BUG #4186: set lc_messages does not work > > > > Bruce Momjian wrote: > > > Tom Lane wrote: > > >> Magnus Hagander <magnus@hagander.net> writes: > > >>> Thomas H. wrote: > > >>>> so at least that explains the "changed" behaviour. nevertheless, > > >>>> LC_MESSAGES seems to be defunct - with the "locale" > > folder present, > > >>>> pg always picks the os' language and ignores the > > lc_message value. > > >>> This looks like I can reproduce though, at least on cvs head. Did > > >>> this work for you in previous versions? > > >> Maybe we were using a different build of gettext in the previous > > >> releases, one that didn't look at the same info as the > > current code? > > >> > > >> Anyway the patch mentioned at the start of the thread > > >> http://archives.postgresql.org/pgsql-patches/2008-02/msg00038.php > > >> purports to fix this. It doesn't seem to have gotten reviewed > > >> though. > > > > > > Agreed. Magnus, someone, can we get feedback on the patch > > at this URL? > > > > > > > > http://archives.postgresql.org/pgsql-patches/2008-02/msg00038.php > > > > IIRC, there was further work to be done on the patch before > > it was to be applied, and we held off the review until then. > > > > Gevik - can you comment on this? Where are we, what needs ot > > be done still? > > > > //Magnus > > -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian wrote: > Tom Lane wrote: >> Magnus Hagander <magnus@hagander.net> writes: >>> Thomas H. wrote: >>>> so at least that explains the "changed" behaviour. nevertheless, >>>> LC_MESSAGES seems to be defunct - with the "locale" folder present, >>>> pg always picks the os' language and ignores the lc_message value. >>> This looks like I can reproduce though, at least on cvs head. Did this >>> work for you in previous versions? >> Maybe we were using a different build of gettext in the previous >> releases, one that didn't look at the same info as the current code? >> >> Anyway the patch mentioned at the start of the thread >> http://archives.postgresql.org/pgsql-patches/2008-02/msg00038.php >> purports to fix this. It doesn't seem to have gotten reviewed >> though. > > Where are we on this? Saito-san and I have been working on this item. The attached patch is for MSVC version and does the following when built under vc8 or later version MSVC. 1. Accepts Windows or ISO style locale name. 2. _putenv the ISO style locale name as LC_MESSAGES environemnt variable so as to be referenced by the gettext library. Note that we have to call _putenv() in msvcrt.dll not in each MSVC version's runtime library msvcrxx.dll. 3. Calls SetThreadLocale() if necessary. The currently used libintl3.dll(0.14.4.1592) seems to need this handling. ISTM it is badly built and ignores the setting of step 2. regards, Hiroshi Inoue Index: pg_locale.c =================================================================== RCS file: /projects/cvsroot/pgsql/src/backend/utils/adt/pg_locale.c,v retrieving revision 1.42 diff -c -c -r1.42 pg_locale.c *** pg_locale.c 23 Sep 2008 09:20:36 -0000 1.42 --- pg_locale.c 7 Dec 2008 15:53:58 -0000 *************** *** 101,112 **** --- 101,303 ---- * LC_XXX variables have been set correctly. (Thank you Perl for making this * kluge necessary.) */ + + #if defined(WIN32) && defined(_MSC_VER) + typedef int (_cdecl *PUTENVPROC)(const char *); + #if defined(LC_MESSAGES) && (_MSC_VER >= 1400) + #define _TYPE_LOCALE_T_AVAILABLE + #include <shlwapi.h> + typedef const char * (_cdecl *NLLOCALENAMEPOSIXPROC)(int, const char *); + + /* + * Never use DLLGETVERSIONPROC typedef'd in Shlwapi.h. + * It's problematic and would cause a crash. + */ + typedef HRESULT (_cdecl *DLLGETVERSIONFUNC)(DLLVERSIONINFO *); + + static char get_lang[64] = ""; + static char get_country[64] = ""; + static LCID glcid = (LCID) -1; + + static BOOL CALLBACK lclist(LPTSTR lcname) + { + static char tmp_country[128] = ""; + DWORD lcid; + char llang[32], lcountry[32]; + + sscanf_s(lcname, "%x", &lcid); + GetLocaleInfoA(lcid, LOCALE_SISO639LANGNAME, llang, sizeof(llang)); + if (0 != _stricmp(llang, get_lang)) + return TRUE; + GetLocaleInfoA(lcid, LOCALE_SISO3166CTRYNAME, lcountry, sizeof(lcountry)); + if ('\0' == get_country[0]) + { + if (SUBLANG_DEFAULT == SUBLANGID(LANGIDFROMLCID(lcid))) + { + glcid = lcid; + return FALSE; + } + return TRUE; + } + if (0 == _stricmp(lcountry, get_country)) + { + glcid = lcid; + return FALSE; + } + return TRUE; + } + #endif /* defined(LC_MESSAGES) && (_MSC_VER >= 1400) */ + + /* + * This function can accept Windows or ISO style locale name. + * Calls SetThreadLocale() if neccesary. + * Sets the ISO style LC_MESSAGES environment variable using + * _putenv() in msvcrt.dll which may be referenced by gettext. + * Returns ISO style LC_MESSAGES locale name. + */ + static char *adjust_LC_MESSAGES(const char *locale, char *envbuf, char *new_locale) + { + #ifdef _TYPE_LOCALE_T_AVAILABLE + char *rtn_locale = locale; + bool isClocale = false; + LCID lcid = (LCID) -1; + int usecategory = LC_CTYPE; + _locale_t loct = NULL; + + if (0 == stricmp("c", locale) || + 0 == stricmp("posix", locale)) + isClocale = true; + if (isClocale) + loct = _create_locale(usecategory, "C"); + else + loct = _create_locale(usecategory, locale); + + if (NULL != loct) + { + lcid = loct->locinfo->lc_handle[usecategory]; + if (0 == lcid) + lcid = MAKELCID(MAKELANGID(LANG_ENGLISH, SUBLANG_ENGLISH_US), SORT_DEFAULT); + _free_locale(loct); + } + else + { + char del[16]; + int scount = sscanf_s(locale, "%[^_-.]%[_-.]%[^.]", get_lang, sizeof(get_lang), del, sizeof(del), get_country,sizeof(get_country)); + switch (scount) + { + case 1: + case 2: + get_country[0] = '\0'; + break; + case 3: + if ('.' == del[0]) + get_country[0] = '\0'; + break; + } + glcid = (LCID) -1; + EnumSystemLocales(lclist, LCID_SUPPORTED); + lcid = glcid; + } + if ((LCID) -1 == lcid) + return NULL; + else + { + /* + * Though GNU gettext FAQ says setting the environtment LC_MESSAGES etc variables + * determines the locale used by gettext, it doesn't seem true for 0.14.4.1952 + * binary version of libintl3.dll. To make sure we do a test and set not only the + * LC_MESSAGES variable but also the current thread's locale if the test shows that + * the DLL doesn't follow the specification(FAQ). + */ + static bool initCheck = true, setThread = false, setEnv = true; + HMODULE hModule; + DLLGETVERSIONFUNC DllGetVersion; + + if (initCheck && + NULL != (hModule = LoadLibrary("libintl3.dll")) && + NULL != (DllGetVersion = (DLLGETVERSIONFUNC) GetProcAddress(hModule, "DllGetVersion"))) + { + DLLVERSIONINFO2 dvi; + HRESULT hr; + NLLOCALENAMEPOSIXPROC locale_name_posix; + + dvi.info1.cbSize = sizeof(dvi); + hr = DllGetVersion((DLLVERSIONINFO *) &dvi); + if (SUCCEEDED(hr) && + MAKEDLLVERULL(0, 14, 4, 1952) == dvi.ullVersion && + NULL != (locale_name_posix = (NLLOCALENAMEPOSIXPROC) GetProcAddress(hModule, "_nl_locale_name_posix"))) + { + + /* + * Here we test the DLL's behavior. + * + * Does the function _nl_locale_name_posix() which is + * internally called by gettext() call setlocale()? + */ + if (NULL != locale_name_posix(LC_CTYPE, "ignore_this_value") && + NULL == locale_name_posix(LC_MESSAGES, "ignore_this_value")) + { + setThread = true; + elog(WARNING, "linking a mal-configured libintl3.dll?"); + } + } + } + initCheck = false; + if (NULL != hModule) + FreeLibrary(hModule); + if (setThread) + { + SetThreadLocale(lcid); + } + if (setEnv && !isClocale) + { + char isolang[64], isocrty[64]; + + GetLocaleInfoA(lcid, LOCALE_SISO639LANGNAME, isolang, sizeof(isolang)); + GetLocaleInfoA(lcid, LOCALE_SISO3166CTRYNAME, isocrty, sizeof(isocrty)); + snprintf(new_locale, LC_ENV_BUFSIZE - 1, "%s_%s", isolang, isocrty); + rtn_locale = new_locale; + } + } + #endif /* _TYPE_LOCALE_T_AVAILABLE */ + #if (_MSC_VER >= 1300) + { + /* + * Each MSVC version has its own _putenv() in its runtime library msvcrXX.dll. + * We call _putenv() in msvcrt.dll so as to be referenced by GnuWin32 library. + */ + HMODULE hmodule; + static PUTENVPROC putenvFunc = NULL; + + if (NULL == putenvFunc) + { + if (hmodule = GetModuleHandle("msvcrt"), NULL == hmodule) + return NULL; + putenvFunc = (PUTENVPROC)GetProcAddress(hmodule, "_putenv"); + } + if (NULL == putenvFunc) + return NULL; + snprintf(envbuf, LC_ENV_BUFSIZE - 1, "%s=%s", "LC_MESSAGES", rtn_locale); + if (putenvFunc(envbuf)) + return NULL; + /* Refresh msgid pool maintained by gettext? */ + textdomain("postgres"); + } + #endif /* _MSC_VER >= 1300 */ + + return rtn_locale; + } + #endif /* defined(WIN32) && defined(_MSC_VER) */ + char * pg_perm_setlocale(int category, const char *locale) { char *result; const char *envvar; char *envbuf; + #ifdef LC_MESSAGES + char newlocale[128]; + #endif /* LC_MESSAGES */ #ifndef WIN32 result = setlocale(category, locale); *************** *** 147,152 **** --- 338,347 ---- case LC_MESSAGES: envvar = "LC_MESSAGES"; envbuf = lc_messages_envbuf; + #if defined(WIN32) && defined(_MSC_VER) + if (result = adjust_LC_MESSAGES(result, envbuf, newlocale), NULL == result) + return NULL; + #endif /* WIN32 */ break; #endif case LC_MONETARY: *************** *** 165,170 **** --- 360,366 ---- elog(FATAL, "unrecognized LC category: %d", category); envvar = NULL; /* keep compiler quiet */ envbuf = NULL; + return NULL; break; } *************** *** 182,189 **** if (!SetEnvironmentVariable(envvar, result)) return NULL; if (_putenv(envbuf)) ! return NULL; ! #endif return result; } --- 378,385 ---- if (!SetEnvironmentVariable(envvar, result)) return NULL; if (_putenv(envbuf)) ! result = NULL; ! #endif /* WIN32 */ return result; }
AFAIK we did not discuss this further. LC_MESSAGES and changing locale functionality for functions like to_date on windows remains broken. To summarize the problem: 1) On *nix systems, we use lc_messages and lc_locale both for functions (like to to_date and to_char) and other system messages where locale plays any role. Lc_messages and lc_locale are read from the environment variables. Unfortunately on Windows the meaning of these variables is not the same as on *nix systems, resulting locale assignment behave unexpectedly (broken). 2) On both systems we are using gettext library which is responsible to read and translate to the correct locale. Gettext on *nix and Windows (and MAC) also has different implementations for the same functionality. Not to mention we are using a very old version of gettext on Windows. 3) As I was working on the patch to fix this I encountered places where the usage of LC_MESSAGES and LC_LOCALE was not clear, especially for functions where the functionality of Oracle was mimicked. 4) The entire problem came to light when we started to compile PG with MSVC (8.3+). On 8.2 which was compiled using MinGW (I guess we do not support it anymore) the behavior of lc_messages and lc_locale was also broken but not as visible as with MSVC. 5) To give you an example: On *nix, the locale system acts when the value of lc_messages or lc_locale (environment variables) changes. The expected values are like: "en_US". On Windows on the other hand, en_US has no meaning. It becomes something like "English (United States)", which is ignored by gettext. Leaving us with no alternative than translating the "English (United States)" to "en_US" and force it to gettext which does not work all the time because gettext discards the forced value when it reloads itself. 6) As I mentioned before, I stopped developing a patch for this because this problem is somewhat deeper than it appears. Any attempt to fix this problem will also result changing code for both locale aware functions like to_date and to_char as for when to use lc_locale and lc_messages in the codebase. Any thoughts? Regards, Gevik Babakhani ------------------------------------------------ PostgreSQL NL http://www.postgresql.nl TrueSoftware BV http://www.truesoftware.nl ------------------------------------------------ > -----Original Message----- > From: Bruce Momjian [mailto:bruce@momjian.us] > Sent: Sunday, December 07, 2008 4:26 AM > To: Gevik Babakhani > Cc: 'Magnus Hagander'; 'Tom Lane'; 'Thomas H.'; > pgsql-bugs@postgresql.org > Subject: Re: [BUGS] BUG #4186: set lc_messages does not work > > Gevik Babakhani wrote: > > My apologies on this late reply. > > The way LC_MESSAGES is handled on windows is much less > efficient and faulty. > > While ago I started with a patch to fix some of the issues I > > encountered on windows and LC_MESSAGES. But I stopped > working on that > > patch because this problem needed to be fixed on many other > places. In > > Windows, handling LC_MESSAGES will not work the same way as *nix > > systems, forcing us to make ugly workarounds. (as I > actually was doing > > with my patch) > > > > To my opinion, unless we think of a coherent solution for handling > > LC_MESSAGES/locale for both *nix and win32 platforms, fixing > > lc_messages and locale issues would break more than fixing it. > > > > BTW: The gettext library we are using on win32 is a very old one. > > This is the most recent posting on the topic, I think. Status? > > -------------------------------------------------------------- > ------------- > > > > > > Regards, > > Gevik. > > > > > > > -----Original Message----- > > > From: Magnus Hagander [mailto:magnus@hagander.net] > > > Sent: Tuesday, August 05, 2008 4:54 PM > > > To: Bruce Momjian > > > Cc: Tom Lane; Thomas H.; pgsql-bugs@postgresql.org; Gevik > Babakhani > > > Subject: Re: [BUGS] BUG #4186: set lc_messages does not work > > > > > > Bruce Momjian wrote: > > > > Tom Lane wrote: > > > >> Magnus Hagander <magnus@hagander.net> writes: > > > >>> Thomas H. wrote: > > > >>>> so at least that explains the "changed" behaviour. > > > >>>> nevertheless, LC_MESSAGES seems to be defunct - with > the "locale" > > > folder present, > > > >>>> pg always picks the os' language and ignores the > > > lc_message value. > > > >>> This looks like I can reproduce though, at least on cvs head. > > > >>> Did this work for you in previous versions? > > > >> Maybe we were using a different build of gettext in > the previous > > > >> releases, one that didn't look at the same info as the > > > current code? > > > >> > > > >> Anyway the patch mentioned at the start of the thread > > > >> > http://archives.postgresql.org/pgsql-patches/2008-02/msg00038.php > > > >> purports to fix this. It doesn't seem to have gotten reviewed > > > >> though. > > > > > > > > Agreed. Magnus, someone, can we get feedback on the patch > > > at this URL? > > > > > > > > > > > http://archives.postgresql.org/pgsql-patches/2008-02/msg00038.php > > > > > > IIRC, there was further work to be done on the patch > before it was > > > to be applied, and we held off the review until then. > > > > > > Gevik - can you comment on this? Where are we, what needs > ot be done > > > still? > > > > > > //Magnus > > > > > -- > Bruce Momjian <bruce@momjian.us> http://momjian.us > EnterpriseDB http://enterprisedb.com > > + If your life is a hard drive, Christ can be your backup. + >