Brar Piening <brar@gmx.de> writes:
> Citing from the Unicode FAQ again:
> Q: Where is a BOM useful?
> A: A BOM is useful at the beginning of files that are typed as text, but
> for which it is not known whether they are in big or little endian
> format�it can also serve as a hint indicating that the file is in
> Unicode, as opposed to in a legacy encoding and furthermore, it act as a
> signature for the specific encoding form used.
Note that the reference to byte order betrays the implicit context
assumption: that we're talking about UTF16 or UTF32 representation.
A BOM in UTF8 data is useless for its intended purpose of disambiguating
byte order. It could possibly be useful for telling UTF8 data apart
from non-UTF8 data, except for the inconvenient fact that that byte
sequence is not invalid data in non-UTF8 encodings.
BOM is useless in UTF8, no matter what Microsoft thinks. Any tool that
relies on it to detect UTF8 data has to have a workaround for overriding
that detection, or it's broken to the point of uselessness.
regards, tom lane