Re: Why "copy ... from stdio" does not return immediately when reading invalid data?

On 02/02/11 10:20 AM, Nicolas Grilly wrote:
> Is the copy protocol (aka PQputCopyData and PQputCopyEnd) designed to
> send gigabytes of data with just one "copy ... from stdio" query, and
> is there a way to be notified of a potential error before calling
> PQputCopyEnd? Or do I have to send my data in small chunks (for
> example batch of 10000 rows), issue a PQputCopyEnd, check for errors,
> and continue with the next chunk?

I would batch the data, maybe 1000 lines or even 100 lines at a time if
these errors are at all frequent.  put the errored batches in an
exception list or something so you can sort them out later.