Hi Ken,
On Tue, 2017-03-07 at 15:20 -0800, Ken Tanzer wrote:
> Hi. I've got a recurring problem with character encoding for a
> Postgres-based web PHP app, and am hoping someone can clue me in or
> at least point me in the right direction. I'll confess upfront my
> understanding of encoding issues is extremely limited. Here goes.
>
> The app uses a Postgres database, UTF-8 encoded. Through their
> browsers, users can add and edit records often including text. Most
> of the time this works fine. Though sometimes this will fail with
> Postgres complaining, for example, "Could query with ... , The error
> text was: ERROR: invalid byte sequence for encoding "UTF8": 0xe9 0x20
> 0x67"
>
> So this generally happens when people copy and paste things out of
> their word documents and such.
>
> As I understand it, those are likely encoded in something non-UTF-8,
> like WIN-1251 or something. And that one way or another, the
> encoding needs to be translated before it can be placed into the
> database. I'm not clear how this is supposed to happen though.
> Automatically by the browser? Done in the app? Some other way? And
> if in the app, how is one supposed to know what the incoming encoding
> is?
>
> Thanks in advance for any help or pointers.
>
> Ken
>
>
>
1) Make sure the text editor you use to create your pages, etc. uses
UTF-8 as its encoding. That way the file's BOM is set correctly.
2) Make sure your headers contain the following:-
<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
or
<meta charset="UTF-8"/> which is HTML5, however the long version is
still recognised by HTML5.
I understand that some versions of IE have issues with correctly
determining the character set, so, unfortunately, you still have to
verify that user input is UTF-8 compatible.
HTH,
Rob