Re: buildfarm / handling (undefined) locales

On 13.5.2014 20:58, Tom Lane wrote:
> Tomas Vondra <tv@fuzzy.cz> writes:
>> a few days ago I switched magpie into an LXC container, and while
>> fixinig unrelated issue there, I noticed that although the tests seemed
>> to run, some of the results are actually rubbish because of missing locales.
> 
>> So for example when the system misses cs_CZ.WIN-1250 (i.e. czech locale
>> with windows codepage 1250), initdb actually did this
> 
>>     The files belonging to this database system will be owned by user
>>     "pgbuild".
>>     This user must also own the server process.
> 
>>     initdb: invalid locale name ""
>>     initdb: invalid locale name ""
>>     initdb: invalid locale name ""
>>     initdb: invalid locale name ""
>>     initdb: invalid locale name ""
>>     initdb: invalid locale name ""
>>     The database cluster will be initialized with locale "C".
>>     The default database encoding has accordingly been set to
>>     "SQL_ASCII".
>>     The default text search configuration will be set to "english".
> 
> Hm, I'm a bit confused as to what you actually did here.  The "invalid
> locale name" bleats seem to indicate that no --locale or --lc_xxx options
> were given on the command line; correct?  If so the issue is presumably
> that the environment variable(s) were set to incorrect values.  While
> we *could* abort in that situation, I've never heard of any program
> that did; the normal response is to silently ignore the environment
> variables and use C locale.  We're not being exactly silent about it
> but I think the outcome is the expected one.
> 
> There is a comment in the code about this:
> 
>     /* should we exit here? */
>     if (res == NULL)
>         fprintf(stderr, _("%s: invalid locale name \"%s\"\n"),
>                 progname, locale);
> 
> but I think what's being questioned is whether an incorrect locale
> name *given as a command line argument* should result in an abort.
> That might be a good idea, but it looks like it'd require some
> restructuring of the code to make it possible to distinguish the
> case from bad-environment.

Hmmmm. Actually the logs above were generated "manually" by setting LANG
to an incorrect value on my desktop machine, i.e. something like this:
  export LANG="cs_CZ.WIN-1250"  pg_ctl -D /tmp/test init

I did this because the logs from the buildfarm animal contain the
messages in czech, which makes them rather unsuitable for english speakers:


http://www.pgbuildfarm.org/cgi-bin/show_stage_log.pl?nm=magpie&dt=2014-04-17%2000%3A07%3A36&stg=initdb-cs_CZ.WIN-1250

I suppose that's why the locale name is empty (it simply gets empty
string from the environment), while the buildfarm client uses a
command-line option to pass the name.

Also, I now noticed that while the example I posted has to C.SQL_ASCII,
the buildfarm then uses cs_CZ.UTF-8. I guess this possible thanks to the
'--locale' option.

If it was up to me, I'd vote to fail in such cases. I find it confusing
(and a bit 'automagic') to receive invalid locale but decide 'the user
surely meant C with SQL_ASCII'. I've actually had to deal with a few
production installations that were installed like this, and it was PITA
to fix.

Anyway, that's not what I was proposing here. I merely think we should
do a simple 'did we actually get the right locale' check somewhere in
the buildfarm client.

>> Yeah, not really what we were shooting for. I've fixed this by defining
>> the missing locales, and indeed - magpie now fails in plpython tests.
> 
> I saw that earlier today (tho right now the buildfarm server seems to
> not be responding :-().  Probably we should use some more-widely-used
> character code in that specific test?

What do you mean by 'not responding'? The tests were not running for ~2
days because of LXC/SELinux issue, causing cron failures.

Anyway, using a more-widely used character is probably the best fix
possible here.

regards
Tomas