Обсуждение: (patch) regression diffs on collate.linux.utf8 test

On master, I see a minor test error (at least on my machine) as well as
a diff. Patch attached.

Regards,
    Jeff Davis

Jeff Davis <pgsql@j-davis.com> writes:
> On master, I see a minor test error (at least on my machine) as well as
> a diff. Patch attached.

Hmm, yeah, I forgot to fix this regression test when I added that DETAIL
line.  However, I don't see the need for fooling with the lc_time value?
        regards, tom lane

On Sun, 2011-10-16 at 16:00 -0400, Tom Lane wrote:
> Jeff Davis <pgsql@j-davis.com> writes:
> > On master, I see a minor test error (at least on my machine) as well as
> > a diff. Patch attached.
>
> Hmm, yeah, I forgot to fix this regression test when I added that DETAIL
> line.  However, I don't see the need for fooling with the lc_time value?
>
>             regards, tom lane

Here is the diff that I'm seeing on master right now with:

  make -s check EXTRA_TESTS=collate.linux.utf8

If I qualify it as "tr_TR.UTF-8" it works. Perhaps I have something
misconfigured on my system (Ubuntu 11.10)? I just installed:
  language-pack-de
  language-pack-tr
  language-pack-sv

in an attempt to make the test work, and it works all except for that
lc_time settng.

Regards,
    Jeff Davis

Jeff Davis <pgsql@j-davis.com> writes:
> On Sun, 2011-10-16 at 16:00 -0400, Tom Lane wrote:
>> Hmm, yeah, I forgot to fix this regression test when I added that DETAIL
>> line.  However, I don't see the need for fooling with the lc_time value?

> Here is the diff that I'm seeing on master right now with:
>   make -s check EXTRA_TESTS=collate.linux.utf8

> If I qualify it as "tr_TR.UTF-8" it works. Perhaps I have something
> misconfigured on my system (Ubuntu 11.10)? I just installed:
>   language-pack-de
>   language-pack-tr
>   language-pack-sv

That's very strange.  It works as-is for me on Fedora 13, 14, and 15,
which you'd expect to have essentially the same I18N infrastructure as
Ubuntu (and I'm pretty sure I didn't install any optional language
support for Turkish).  Anybody know what the problem is?

The reason I'm resisting just changing it is that I'd prefer to minimize
the number of dependencies this regression test has on the exact
spelling of "UTF-8", as that is not terribly well standardized.  On my
Fedora boxes, for instance, locale -a says that "tr_TR.utf8" is the
name of that particular locale.
        regards, tom lane

On tis, 2011-10-18 at 01:07 -0700, Jeff Davis wrote:
> On Sun, 2011-10-16 at 16:00 -0400, Tom Lane wrote:
> > Jeff Davis <pgsql@j-davis.com> writes:
> > > On master, I see a minor test error (at least on my machine) as well as
> > > a diff. Patch attached.
> > 
> > Hmm, yeah, I forgot to fix this regression test when I added that DETAIL
> > line.  However, I don't see the need for fooling with the lc_time value?
> > 
> >             regards, tom lane
> 
> Here is the diff that I'm seeing on master right now with:
> 
>   make -s check EXTRA_TESTS=collate.linux.utf8
> 
> If I qualify it as "tr_TR.UTF-8" it works. Perhaps I have something
> misconfigured on my system (Ubuntu 11.10)? I just installed:
>   language-pack-de
>   language-pack-tr
>   language-pack-sv
> 
> in an attempt to make the test work, and it works all except for that
> lc_time settng.

I think the language-pack packages have nothing to do with it; they only
supply translations.

Possibly, things are set up so that only UTF-8 locales are installed by
default.  Since the collate.linux.utf8 requires a UTF-8 environment, it
seems reasonable to use the tr_TR.UTF-8 locale for LC_TIME, instead of
requiring an unrelated (ISO-8859-9) locale to be installed.  So I think
the change you propose is reasonable.

Peter Eisentraut <peter_e@gmx.net> writes:
> On tis, 2011-10-18 at 01:07 -0700, Jeff Davis wrote:
>> If I qualify it as "tr_TR.UTF-8" it works. Perhaps I have something
>> misconfigured on my system (Ubuntu 11.10)? I just installed:
>> language-pack-de
>> language-pack-tr
>> language-pack-sv
>> in an attempt to make the test work, and it works all except for that
>> lc_time settng.

> I think the language-pack packages have nothing to do with it; they only
> supply translations.

> Possibly, things are set up so that only UTF-8 locales are installed by
> default.  Since the collate.linux.utf8 requires a UTF-8 environment, it
> seems reasonable to use the tr_TR.UTF-8 locale for LC_TIME, instead of
> requiring an unrelated (ISO-8859-9) locale to be installed.  So I think
> the change you propose is reasonable.

As I said to Jeff earlier, I'd rather not embed assumptions about the
spelling of encoding names into this test.  So I don't want to do this
just to get rid of an unexplained failure.  I don't entirely believe
the above theory, because it's not clear why Jeff's machine is behaving
differently from mine.
        regards, tom lane

On tis, 2011-10-18 at 15:21 -0400, Tom Lane wrote:
> Peter Eisentraut <peter_e@gmx.net> writes:
> > On tis, 2011-10-18 at 01:07 -0700, Jeff Davis wrote:
> >> If I qualify it as "tr_TR.UTF-8" it works. Perhaps I have something
> >> misconfigured on my system (Ubuntu 11.10)? I just installed:
> >> language-pack-de
> >> language-pack-tr
> >> language-pack-sv
> >> in an attempt to make the test work, and it works all except for that
> >> lc_time settng.
> 
> > I think the language-pack packages have nothing to do with it; they only
> > supply translations.
> 
> > Possibly, things are set up so that only UTF-8 locales are installed by
> > default.  Since the collate.linux.utf8 requires a UTF-8 environment, it
> > seems reasonable to use the tr_TR.UTF-8 locale for LC_TIME, instead of
> > requiring an unrelated (ISO-8859-9) locale to be installed.  So I think
> > the change you propose is reasonable.
> 
> As I said to Jeff earlier, I'd rather not embed assumptions about the
> spelling of encoding names into this test.  So I don't want to do this
> just to get rid of an unexplained failure.  I don't entirely believe
> the above theory, because it's not clear why Jeff's machine is behaving
> differently from mine.

Presumably because Jeff doesn't have that particular locale installed.
locale -a would clarify that.

glibc has always accepted variant locale name spellings such as "UTF-8"
vs "utf8", so it's not a problem.

On Tue, 2011-10-18 at 22:25 +0300, Peter Eisentraut wrote:
> Presumably because Jeff doesn't have that particular locale installed.
> locale -a would clarify that.

$ locale -a |grep -i tr
tr_CY.utf8
tr_TR.utf8

So, yes, I only have the UTF8 version. I didn't realize they were
different -- do you happen to know what package I need for just plain
tr_TR?

Regards,Jeff Davis

Jeff Davis <pgsql@j-davis.com> writes:
> On Tue, 2011-10-18 at 22:25 +0300, Peter Eisentraut wrote:
>> Presumably because Jeff doesn't have that particular locale installed.
>> locale -a would clarify that.

> $ locale -a |grep -i tr
> tr_CY.utf8
> tr_TR.utf8

> So, yes, I only have the UTF8 version.

Wow, that's interesting.  Digging around on my Fedora box, I can't find
any suggestion that it's even possible to subdivide the locale settings
like that.  I only see one source file for tr_TR --- that's
/usr/share/i18n/locales/tr_TR --- and it looks like all the stuff under 
/usr/share/i18n/locales/ is compiled into one big run-time file
/usr/lib/locale/locale-archive.
        regards, tom lane

On tis, 2011-10-18 at 21:47 -0700, Jeff Davis wrote:
> On Tue, 2011-10-18 at 22:25 +0300, Peter Eisentraut wrote:
> > Presumably because Jeff doesn't have that particular locale installed.
> > locale -a would clarify that.
> 
> $ locale -a |grep -i tr
> tr_CY.utf8
> tr_TR.utf8
> 
> So, yes, I only have the UTF8 version. I didn't realize they were
> different -- do you happen to know what package I need for just plain
> tr_TR?

dpkg-reconfigure locales

or

apt-get install locales-all

On ons, 2011-10-19 at 01:10 -0400, Tom Lane wrote:
> Jeff Davis <pgsql@j-davis.com> writes:
> > On Tue, 2011-10-18 at 22:25 +0300, Peter Eisentraut wrote:
> >> Presumably because Jeff doesn't have that particular locale installed.
> >> locale -a would clarify that.
> 
> > $ locale -a |grep -i tr
> > tr_CY.utf8
> > tr_TR.utf8
> 
> > So, yes, I only have the UTF8 version.
> 
> Wow, that's interesting.  Digging around on my Fedora box, I can't find
> any suggestion that it's even possible to subdivide the locale settings
> like that.  I only see one source file for tr_TR --- that's
> /usr/share/i18n/locales/tr_TR --- and it looks like all the stuff under 
> /usr/share/i18n/locales/ is compiled into one big run-time file
> /usr/lib/locale/locale-archive.

It has "always" been the case on Debian that it doesn't blindly install
all 600+ locales provided by glibc.  Instead, the OS installer picks the
ones that you are likely to use, and generates those from source at the
time the "locales" package is installed.  (So the locales package
contains the source for all glibc locales, but not the binary form.)

In fact, here is the output from a vanilla Debian stable installation:

$ locale -a
C
en_US.utf8
POSIX

I suspect, and this is Ubuntu-specific, so I don't have direct
experience with it, that what happened is that when you install the
langpack packages that Jeff mentioned, it triggers the compilation of
the respective associated locales.  But as you can see, apparently only
utf8 locales are generated by default, nowadays.

On Wed, 2011-10-19 at 11:44 +0300, Peter Eisentraut wrote:
> On tis, 2011-10-18 at 21:47 -0700, Jeff Davis wrote:
> > On Tue, 2011-10-18 at 22:25 +0300, Peter Eisentraut wrote:
> > > Presumably because Jeff doesn't have that particular locale installed.
> > > locale -a would clarify that.
> > 
> > $ locale -a |grep -i tr
> > tr_CY.utf8
> > tr_TR.utf8
> > 
> > So, yes, I only have the UTF8 version. I didn't realize they were
> > different -- do you happen to know what package I need for just plain
> > tr_TR?
> 
> dpkg-reconfigure locales

Did that, and still:

# locale -a|grep -i tr
tr_CY.utf8
tr_TR.utf8

> apt-get install locales-all

# aptitude install locales-all
No candidate version found for locales-all
No candidate version found for locales-all
No packages will be installed, upgraded, or removed.
0 packages upgraded, 0 newly installed, 0 to remove and 80 not upgraded.
Need to get 0 B of archives. After unpacking 0 B will be used.                                        
Regards,Jeff Davis

On Wed, 2011-10-19 at 10:10 -0700, Jeff Davis wrote:
> > dpkg-reconfigure locales

I had to manually do

# locale-gen tr_TR

to make it generate tr_TR.ISO-8859-9, and now it works.

I'm not sure what we should do, exactly, but I expect that others who
attempt to run the test on ubuntu (and maybe debian) might get confused.
I'd be fine with leaving it alone though, too.

Regards,Jeff Davis