Обсуждение: Charset conversion error

Hi, I have an error after updating my database up to 8.1.4 version.
"SQL Error: ERROR:  character 0xb9 of encoding "WIN1251" has no equivalent in "MULE_INTERNAL"'. "
My client program encoding is windows-1251 and database encoding is koi8.
What can I do to rectify the situation?

Verba volent, scripta manent
Dan Black

> Hi, I have an error after updating my database up to 8.1.4 version.
> "SQL Error: ERROR:  character 0xb9 of encoding "WIN1251" has no equivalent
> in "MULE_INTERNAL"'. "
> My client program encoding is windows-1251 and database encoding is koi8.
> What can I do to rectify the situation?

It suggests that Windows-1251's 0xb9 cannot be converted to KOI8. You
should check your Windows-1251 data.
--
Tatsuo Ishii
SRA OSS, Inc. Japan

I think you are right. But everything was alright before I updated my database.
So there are a lot of "incorrect" values in tables. And errors appear when I execute "SELECT * FROM table".




--
Verba volent, scripta manent
Dan Black

> I think you are right. But everything was alright before I updated my
> database.

Previous version converted such characters to ASCII spaces. So
probably you have lots of bogus spaces anyway. If you think it's ok,
then you could your own CONVERSION which behaves similar to previous
version.
--
Tatsuo Ishii
SRA OSS, Inc. Japan

> So there are a lot of "incorrect" values in tables. And errors appear when I
> execute "SELECT * FROM table".
>
> 2006/5/30, Tatsuo Ishii <ishii@sraoss.co.jp>:
> >
> > > Hi, I have an error after updating my database up to 8.1.4 version.
> > > "SQL Error: ERROR:  character 0xb9 of encoding "WIN1251" has no
> > equivalent
> > > in "MULE_INTERNAL"'. "
> > > My client program encoding is windows-1251 and database encoding is
> > koi8.
> > > What can I do to rectify the situation?
> >
> > It suggests that Windows-1251's 0xb9 cannot be converted to KOI8. You
> > should check your Windows-1251 data.
> > --
> > Tatsuo Ishii
> > SRA OSS, Inc. Japan
> >
>
>
>
> --
> Verba volent, scripta manent
> Dan Black

Previous version converted such characters all right. And there wasn't any bogus ASCII spaces. But I looked at the KOI8 charset table and found out that there is not equivalent symbol (0xb9) in this table.




--
Verba volent, scripta manent
Dan Black

As far as I know the conversion table was not changed. Are you saying
that Win-1251 0xb9 is converted to a non-space-char in the previous
version?
--
Tatsuo Ishii
SRA OSS, Inc. Japan

> Previous version converted such characters all right. And there wasn't any
> bogus ASCII spaces. But I looked at the KOI8 charset table and found out
> that there is not equivalent symbol (0xb9) in this table.
>
> 2006/5/30, Tatsuo Ishii <ishii@sraoss.co.jp>:
> >
> > > I think you are right. But everything was alright before I updated my
> > > database.
> >
> > Previous version converted such characters to ASCII spaces. So
> > probably you have lots of bogus spaces anyway. If you think it's ok,
> > then you could your own CONVERSION which behaves similar to previous
> > version.
> > --
> > Tatsuo Ishii
> > SRA OSS, Inc. Japan
> >
> > > So there are a lot of "incorrect" values in tables. And errors appear
> > when I
> > > execute "SELECT * FROM table".
> > >
> > > 2006/5/30, Tatsuo Ishii <ishii@sraoss.co.jp>:
> > > >
> > > > > Hi, I have an error after updating my database up to 8.1.4 version.
> > > > > "SQL Error: ERROR:  character 0xb9 of encoding "WIN1251" has no
> > > > equivalent
> > > > > in "MULE_INTERNAL"'. "
> > > > > My client program encoding is windows-1251 and database encoding is
> > > > koi8.
> > > > > What can I do to rectify the situation?
> > > >
> > > > It suggests that Windows-1251's 0xb9 cannot be converted to KOI8. You
> > > > should check your Windows-1251 data.
> > > > --
> > > > Tatsuo Ishii
> > > > SRA OSS, Inc. Japan
> > > >
> > >
> > >
> > >
> > > --
> > > Verba volent, scripta manent
> > > Dan Black
> >
>
>
>
> --
> Verba volent, scripta manent
> Dan Black

It seems not.

$ cat koi8.sh
printf "SELECT '==\xb9=='"|PGCLIENTENCODING=WIN1251 psql -p 5435 koi8

$ psql -p 5435 -c 'select version()' koi8
                                                 version
---------------------------------------------------------------------------------------------------------
 PostgreSQL 8.1.3 on i686-pc-linux-gnu, compiled by GCC gcc (GCC) 3.3.2 20031218 (Vine Linux 3.3.2-0vl8)
(1 row)

$ sh koi8.sh
 ?column?
----------
 == ==
(1 row)

As you can see PostgreSQL 8.1.3 converts 0xb9 in Windows-1251 to a
space of KOI8.
--
Tatsuo Ishii
SRA OSS, Inc. Japan

> Yes, character converted to a non-space-char.
> And there is reverse error. It's happened when I try to extract information
> from database.
> SQL Error: ERROR:  character 0x8bbf of encoding "MULE_INTERNAL" has no
> equivalent in "WIN1251""
>
> 2006/5/30, Tatsuo Ishii <ishii@sraoss.co.jp>:
> >
> > As far as I know the conversion table was not changed. Are you saying
> > that Win-1251 0xb9 is converted to a non-space-char in the previous
> > version?
> > --
> > Tatsuo Ishii
> > SRA OSS, Inc. Japan
> >
> > > Previous version converted such characters all right. And there wasn't
> > any
> > > bogus ASCII spaces. But I looked at the KOI8 charset table and found out
> > > that there is not equivalent symbol (0xb9) in this table.
> > >
> > > 2006/5/30, Tatsuo Ishii <ishii@sraoss.co.jp>:
> > > >
> > > > > I think you are right. But everything was alright before I updated
> > my
> > > > > database.
> > > >
> > > > Previous version converted such characters to ASCII spaces. So
> > > > probably you have lots of bogus spaces anyway. If you think it's ok,
> > > > then you could your own CONVERSION which behaves similar to previous
> > > > version.
> > > > --
> > > > Tatsuo Ishii
> > > > SRA OSS, Inc. Japan
> > > >
> > > > > So there are a lot of "incorrect" values in tables. And errors
> > appear
> > > > when I
> > > > > execute "SELECT * FROM table".
> > > > >
> > > > > 2006/5/30, Tatsuo Ishii <ishii@sraoss.co.jp>:
> > > > > >
> > > > > > > Hi, I have an error after updating my database up to 8.1.4version.
> > > > > > > "SQL Error: ERROR:  character 0xb9 of encoding "WIN1251" has no
> > > > > > equivalent
> > > > > > > in "MULE_INTERNAL"'. "
> > > > > > > My client program encoding is windows-1251 and database encoding
> > is
> > > > > > koi8.
> > > > > > > What can I do to rectify the situation?
> > > > > >
> > > > > > It suggests that Windows-1251's 0xb9 cannot be converted to KOI8.
> > You
> > > > > > should check your Windows-1251 data.
> > > > > > --
> > > > > > Tatsuo Ishii
> > > > > > SRA OSS, Inc. Japan
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Verba volent, scripta manent
> > > > > Dan Black
> > > >
> > >
> > >
> > >
> > > --
> > > Verba volent, scripta manent
> > > Dan Black
> >
>
>
>
> --
> Verba volent, scripta manent
> Dan Black

You are right. I was in a harry when wrote my last message.
I solved my problem by changing some source files. But It is not very good because I have to do it every time when I need to update my database. And I can't check all fields in all tables because it take a few days.




--
Verba volent, scripta manent
Dan Black