COPY problem.

I am seeing the following error:

ERROR:  end-of-copy marker corrupt
CONTEXT:  COPY moreover_documents_y2010m06, line 49327

What is that? The code that produces the error above is extremely simple:

sub do_copy
{
    $dbh->do("COPY $table FROM STDIN WITH NULL AS E'\e'");
    while (<>)
    {
        die($dbh->errstr . "\n") if ($dbh->pg_putcopydata($_) != 1);
    }
    $dbh->pg_putcopyend();
    $dbh->commit();
}

I am copying the textual data, provided by the following routine:

my $FILE  = "|write_table -t $dest -n $ncols";
$FILE .= " -c" if defined($copy);
open(STDOUT, $FILE) || die("Cannot open output:$!\n");

while (my @row = $sth->fetchrow_array())
{
    map { $_ = "\e" if !defined($_); s/[\t\n]+/ /g; } @row;
    my $line = join $sep, @row;
    print "$line\n";
    --$nrows;
    last if ($nrows == 0);
}

The $sth is a SQL statement handle which selects from another database.
There are several millions of rows and it is hard to figure out which
one has failed. I am trying with a normal SQL insert, but that is much
slower.


--

Mladen Gogala
Sr. Oracle DBA
1500 Broadway
New York, NY 10036
(212) 329-5251
http://www.vmsinfo.com
The Leader in Integrated Media Intelligence Solutions