Обсуждение: Error in running DBT2

Rohit Goyal wrote
> Hi Peter,
> 
> On Tue, May 13, 2014 at 9:44 PM, Peter Geoghegan <

> pg@

> > wrote:
> 
>>
>> On Tue, May 13, 2014 at 12:36 PM, Rohit Goyal <

> rhtgyl.87@

> > wrote:
>>
>>> This pattern the above found many times. Please guide me through!!!
>>>
>>
>> IIRC, people have been working around this by setting
>> standard_conforming_strings to "off". It really ought to be fixed in a
>> principled way, though -- the real issue here is that dbt2 has severe
>> bit-rot.
>>
>> Can you please elaborate, where I can make this changes :) ?

Peter?

The error is more typical of someone creating a database with UTF-8 encoding
but then tries storing Latin-1 (or some other) encoded data.  Since not all
byte sequences in the source data encoding are valid in UTF-8 when one of
the invalid sequences is present the import fails.  PostgreSQL makes no
attempt to perform data conversion on import.

The solution is to create the database with the correct encoding - whatever
it may be.  This "DBT2" application should provide guidance on this topic. 
Otherwise you could use "SQL_ASCII" - which is effectively punting on the
issue:

http://www.postgresql.org/docs/9.3/static/sql-createdatabase.html
http://www.postgresql.org/docs/9.3/static/multibyte.html#MULTIBYTE-CHARSET-SUPPORTED

IIRC given that the error happens in a SQL COPY
"standard_conforming_strings" has no bearing on the outcome.





--
View this message in context: http://postgresql.1045698.n5.nabble.com/Error-in-running-DBT2-tp5803793p5803817.html
Sent from the PostgreSQL - hackers mailing list archive at Nabble.com.

On Tue, May 13, 2014 at 2:51 PM, David G Johnston
<david.g.johnston@gmail.com> wrote:
> The error is more typical of someone creating a database with UTF-8 encoding
> but then tries storing Latin-1 (or some other) encoded data.

I didn't say that it was the only problem. Just that I had noticed
that other people were using standard_conforming_strings = off when
running DBT2, on the couple of occasions were I looked through a
result set. The fact that this may have accidentally worked before,
when that was the default does not imply that it was ever correct, but
it may at least be possible to get it to work at least as well as it
ever did by doing that. DBT2 doesn't seem to have anything to say
about encoding one way or the other. As I said, it has many problems.
Another thing that I noticed that I didn't like was that it uses the C
locale.

-- 
Peter Geoghegan

On 05/20/2014 03:39 AM, Rohit Goyal wrote:
> Hi All,
>
> Just adding the actual error again.
>
> I run the the *dbt2-pgsql-build-db -w 1 *
>
> Output directory of data files: current directory
>
> *Generating data files for 1 warehouse(s)...*
> *Generating item table data...*
> *BEGIN*
> *ERROR:  invalid byte sequence for encoding "UTF8": 0xf9*
> *CONTEXT:  COPY item, line 1*
> *ROLLBACK*
>
> This pattern the above found many times. Please guide me through!!!
>
>
>


Please stop posting this to postgresql-hackers. We've already told you 
it's the wrong forum.

cheers

andrew