Re: Support UTF-8 files with BOM in COPY FROM

On mån, 2011-09-26 at 13:19 -0400, Robert Haas wrote:
> The thing that makes me doubt that is this comment from Tatsuo Ishii:
> 
> TI> COPY explicitly specifies the encoding (to be UTF-8 in this case).
> So
> TI> I think we should not regard U+FEFF as "BOM" in COPY, rather we
> should
> TI> regard U+FEFF as "ZERO WIDTH NO-BREAK SPACE".
> 
> If a BOM is confusable with valid data, then I think recognizing it
> and discarding it unconditionally is no good - you could end up where
> COPY OUT, TRUNCATE, COPY IN changes the table contents.

We did recently accept a patch for psql -f to skip over a UTF-8
byte-order mark.  We had a lot of this same discussion there.