Re: Pg 7.4 to 8.1 UTF problems

Mario Splivalo <msplival@jagor.srce.hr> writes:
> psql:the_database.psql:86: ERROR:  invalid UTF-8 byte sequence detected
> near byte 0x8d
> CONTEXT:  COPY _netsms, line 1367, column text: "Padamo u pozu 69.uz
> \uffffmas mog macana u usta.a ja tvoju ljepu picu"

7.4's checking for valid UTF8 code sequences had some bugs, causing it
to accept data that is not valid UTF8.  8.1 has tightened that up.

> From here it is visible that the troubled charachter has has ASCII HEX
> code 8d, which is regular ascii charachter.

It is not ASCII, and it is not legal UTF8 either, at least not without
another byte >= 0x80 after it.

You need to decide whether this is bad data (and if so fix it), or
whether you misdetermined what the encoding of your data is (and if
so, change to the correct encoding declaration).

            regards, tom lane