Re: pg_dump/pg_restore problems with 7.4.1

Bruce Momjian <pgman@candle.pha.pa.us> writes:
> Ben Marklein wrote:
>> db=# CREATE UNIQUE INDEX person_info_username_ix ON
>> person_info USING btree (username);
>> ERROR:  could not create unique index
>> DETAIL:  Table contains duplicated values.
>>
>> Of course, this index existed in the 7.2 DB, so it
>> seems odd that it should not be possible to recreate.

> Not sure.  Never heard of this before.

It seems odd to me too.  Is it possible that the new database uses a
different character set encoding and/or different locale, such that
two strings previously considered unequal are now equal?  (Of course,
you'd think the count(*) check would find any such cases...)

>> 2) While importing I get: "ERROR:  invalid memory
>> alloc request size 1073741824".  I tracked this down
>> to a line with some non-ASCII characters.

> Sounds like the same problem, but I don't remember the solution.

I think this is this problem:

2004-01-17 21:15  tgl

    * src/backend/commands/: copy.c (REL7_4_STABLE), copy.c: Don't use
    %s-with-precision format spec to truncate data being displayed in a
    COPY error message.  It seems that glibc gets indigestion if it is
    asked to truncate strings that contain invalid UTF-8 encoding
    sequences.  vsnprintf will return -1 in such cases, leading to
    looping and eventual memory overflow in elog.c.  Instead use our
    own, more robust pg_mbcliplen routine.    I believe this problem
    accounts for several recent reports of unexpected 'out of memory'
    errors during COPY IN.

Until we get around to issuing a 7.4.2, the easiest solution is to fix
the data to match the database encoding (or vice versa).  You'd actually
have to do that anyway --- the bug is merely that you get a useless
error message instead of a helpful one when the incoming data is bogus
according to the database encoding.

            regards, tom lane