Обсуждение: pgsql: setlocale() on Windows doesn't work correctly if the locale name

setlocale() on Windows doesn't work correctly if the locale name contains
apostrophes or dots. There isn't much hope of Microsoft fixing it any time
soon, it's been like that for ages, so we better work around it. So, map a
few common Windows locale names known to cause problems to aliases that work.

Branch
------
master

Details
-------
http://git.postgresql.org/pg/commitdiff/d5a7bf8c11c8b66c822bbb1a6c90e1a14425bd6e

Modified Files
--------------
src/bin/initdb/initdb.c |   89 +++++++++++++++++++++++++++++++++++++++++++----
1 files changed, 82 insertions(+), 7 deletions(-)

(2011/04/16 2:56), Heikki Linnakangas wrote:
> setlocale() on Windows doesn't work correctly if the locale name contains
> apostrophes or dots.

As for apostrophes, isn't the cause that initdb loses the single quote
of locale? ([BUGS] BUG #5818: initdb lose the single quote of locale)

As the bug reporter mentions, initdb loses the single quote in reality.
Concretely speaking, scanstr() called from bootscanner.l loses it.
I'm not sure if it's suitable for the bootstrap code to call scanstr().

regards,
Hiroshi Inoue

 > There isn't much hope of Microsoft fixing it any time
> soon, it's been like that for ages, so we better work around it. So, map a
> few common Windows locale names known to cause problems to aliases that work.
>
> Branch
> ------
> master
>
> Details
> -------
> http://git.postgresql.org/pg/commitdiff/d5a7bf8c11c8b66c822bbb1a6c90e1a14425bd6e
>
> Modified Files
> --------------
> src/bin/initdb/initdb.c |   89 +++++++++++++++++++++++++++++++++++++++++++----
> 1 files changed, 82 insertions(+), 7 deletions(-)

Hiroshi Inoue <inoue@tpf.co.jp> writes:
> (2011/04/16 2:56), Heikki Linnakangas wrote:
>> setlocale() on Windows doesn't work correctly if the locale name contains
>> apostrophes or dots.

> As for apostrophes, isn't the cause that initdb loses the single quote
> of locale? ([BUGS] BUG #5818: initdb lose the single quote of locale)

> As the bug reporter mentions, initdb loses the single quote in reality.
> Concretely speaking, scanstr() called from bootscanner.l loses it.
> I'm not sure if it's suitable for the bootstrap code to call scanstr().

Huh?  Bootstrap mode just deals with the data found in
src/include/catalog/*.h.  The locale names found by initdb.c are stuck
in there afterwards, using regular SQL commands.  I don't know where the
problem really comes from, but I doubt the connection you're trying to
make above.

            regards, tom lane

(2011/04/20 9:22), Tom Lane wrote:
> Hiroshi Inoue<inoue@tpf.co.jp>  writes:
>> (2011/04/16 2:56), Heikki Linnakangas wrote:
>>> setlocale() on Windows doesn't work correctly if the locale name contains
>>> apostrophes or dots.
>
>> As for apostrophes, isn't the cause that initdb loses the single quote
>> of locale? ([BUGS] BUG #5818: initdb lose the single quote of locale)
>
>> As the bug reporter mentions, initdb loses the single quote in reality.
>> Concretely speaking, scanstr() called from bootscanner.l loses it.
>> I'm not sure if it's suitable for the bootstrap code to call scanstr().
>
> Huh?  Bootstrap mode just deals with the data found in
> src/include/catalog/*.h.  The locale names found by initdb.c are stuck
> in there afterwards, using regular SQL commands.

bootstrap_template1() in initdb runs the BKI script in bootstrap
mode to create template1. Some symbols (LC_COLLATE, LC_CTYPE in
pg_database etc) in the BKI script are substituted by actual values
using replace_token(). Isn't it correct?
ISTM replace_token() takes care of nothing about single quotes
in its input values but the comment in scanstr() says
                        /*
                         * Note: if scanner is working right, unescaped
quotes can only
                         * appear in pairs, so there should be another
character.
                         */

regards,
Hiroshi Inoue

> I don't know where the
> problem really comes from, but I doubt the connection you're trying to
> make above.
>
>             regards, tom lane

On 04/19/2011 09:42 PM, Hiroshi Inoue wrote:
>
> bootstrap_template1() in initdb runs the BKI script in bootstrap
> mode to create template1. Some symbols (LC_COLLATE, LC_CTYPE in
> pg_database etc) in the BKI script are substituted by actual values
> using replace_token(). Isn't it correct?
> ISTM replace_token() takes care of nothing about single quotes
> in its input values but the comment in scanstr() says
>                         /*
>                          * Note: if scanner is working right, unescaped
> quotes can only
>                          * appear in pairs, so there should be another
> character.
>                          */
>

That's perfectly true, but only one of the replaced locale names
contains a single quote mark. So clearly there's more going on here than
just the bug you're referring to. Heikki's commit message specifically
refers to dots in locale names, which shouldn't cause a problem of that
type, I believe.

cheers

andrew

(2011/04/20 12:25), Andrew Dunstan wrote:
>
> On 04/19/2011 09:42 PM, Hiroshi Inoue wrote:
>>
>> bootstrap_template1() in initdb runs the BKI script in bootstrap
>> mode to create template1. Some symbols (LC_COLLATE, LC_CTYPE in
>> pg_database etc) in the BKI script are substituted by actual values
>> using replace_token(). Isn't it correct?
>> ISTM replace_token() takes care of nothing about single quotes
>> in its input values but the comment in scanstr() says
>>                          /*
>>                           * Note: if scanner is working right, unescaped
>> quotes can only
>>                           * appear in pairs, so there should be another
>> character.
>>                           */
>>
>
> That's perfectly true, but only one of the replaced locale names
> contains a single quote mark. So clearly there's more going on here than
> just the bug you're referring to. Heikki's commit message specifically
> refers to dots in locale names, which shouldn't cause a problem of that
> type, I believe.

Yes it's completely another issue as for dots.
I can find no concrete reference to problems about locale
 names containing dots. Is the following an example?

In my environment (Windows Vista using VC8)

  setlocale(LC_XXXX, "Chinese (Traditional)_MCO.950");
works and
  setlocale(LC_XXXX, NULL);
returns
  Chinese (Traditional)_Macao S.A.R..950
but
  setlocale(LC_XXXX, "Chinese (Traditional)_Macao S.A.R..950");
fails.

regards,
Hiroshi Inoue