Обсуждение: ignoring primary key violations in COPY command

<div class="Section1"><p><font face="Tahoma" size="2"><span style="font-size:10.0pt;font-family:Tahoma">Hi all,
</span></font><p><fontface="Tahoma" size="2"><span style="font-size:10.0pt;font-family:Tahoma">We are importing the
datafrom the CSV file into the database using COPY command.</span></font> <font face="Tahoma" size="2"><span
style="font-size:10.0pt;font-family:Tahoma">Butwhen the</span></font> <font face="Tahoma" size="2"><span
style="font-size:10.0pt;
font-family:Tahoma">‘primary key</span></font> <font face="Tahoma" size="2"><span
style="font-size:10.0pt;font-family:Tahoma">‘violation occurs , it stops </span></font><p><font face="Tahoma"
size="2"><spanstyle="font-size:10.0pt;font-family:Tahoma">Then and there and data is not</span></font> <font
face="Tahoma"size="2"><span style="font-size:10.0pt;font-family:Tahoma">updated into the</span></font> <font
face="Tahoma"size="2"><span style="font-size:10.0pt;font-family:Tahoma">tables.</span></font> <font face="Tahoma"
size="2"><spanstyle="font-size:10.0pt;font-family:Tahoma">Also in the CSV file the number of columns is not fixed , its
varies</span></font><p><font face="Tahoma" size="2"><span style="font-size:10.0pt;font-family:Tahoma">In number , it
canrange anywhere between 1 -22,</span></font> <font face="Tahoma" size="2"><span
style="font-size:10.0pt;font-family:Tahoma">Ineed to figure out a way such that I need to</span></font> <font
face="Tahoma"size="2"><span style="font-size:10.0pt;font-family:Tahoma">update only those columns which are present
</span></font><p><fontface="Tahoma" size="2"><span style="font-size:10.0pt;font-family:Tahoma">In the CSV file . This
ofcourse</span></font>  <font face="Tahoma" size="2"><span style="font-size:10.0pt;font-family:Tahoma">I would like
to</span></font><font face="Tahoma" size="2"><span style="font-size:10.0pt;font-family:Tahoma">accomplish using COPY
command, Please let me know if at all this is possible in postgresql</span></font> <font face="Tahoma" size="2"><span
style="font-size:10.0pt;
font-family:Tahoma">8.1</span></font><p><a name=""><i><font face="Times New Roman" size="3"><span style="font-size:
12.0pt;font-style:italic">Thanks and regards,</span></font></i></a><p><font color="green" face="Times New Roman"
size="2"><spanstyle="font-size:10.0pt; 
color:green">Rajendra Singh</span></font><p><font face="Times New Roman" size="3"><span
style="font-size:12.0pt"> </span></font><p><fontcolor="red" face="Times New Roman" size="2"><span
style="font-size:10.0pt;
color:red">Wipro Technologies</span></font><p><font color="navy" face="Times New Roman" size="2"><span
style="font-size:10.0pt;
color:navy">Plot No. 72, Hosur Main road</span></font><p><font color="navy" face="Times New Roman" size="2"><span
style="font-size:10.0pt;color:navy">Keonics</span></font><fontcolor="navy" size="2"><span
style="font-size:10.0pt;color:navy">Electronics City</span></font><p><font color="navy" face="Times New Roman"
size="2"><spanstyle="font-size:10.0pt;color:navy">Bangalore-</span></font><font color="navy" size="2"><span
style="font-size:10.0pt;color:navy">560100, INDIA</span></font><p><i><font color="#3366ff" face="Times New Roman"
size="3"><spanstyle="font-size:12.0pt;color:#3366FF;font-style:italic">I believe we should all pay our tax with a
smile.I tried - but they wanted cash</span></font></i></div><table><tr><td bgcolor="#ffffff"><font color="#000000"><br
/>The information contained in this electronic message and any attachments to this message are intended for the
exclusiveuse of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not
theintended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately
anddestroy all copies of this message and any attachments. <br /><br /> WARNING: Computer viruses can be transmitted
viaemail. The recipient should check this email and any attachments for the presence of viruses. The company accepts no
liabilityfor any damage caused by any virus transmitted by this email.<br /><br /> www.wipro.com<br
/></font></td></tr></table><table><tr><tdbgcolor="#ffffff"><font color="#000000"><br /> The information contained in
thiselectronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and
maycontain proprietary, confidential or privileged information. If you are not the intended recipient, you should not
disseminate,distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message
andany attachments. <br /><br /> WARNING: Computer viruses can be transmitted via email. The recipient should check
thisemail and any attachments for the presence of viruses. The company accepts no liability for any damage caused by
anyvirus transmitted by this email.<br /><br /> www.wipro.com<br /></font></td></tr></table> 

singh.srajendra@wipro.com wrote:
> Hi all,
> 
> We are importing the data from the CSV file into the database using COPY 
> command. But when the ‘primary key ‘ violation occurs , it stops
> 
> Then and there and data is not updated into the tables. Also in the CSV 
> file the number of columns is not fixed , its varies
> 
> In number , it can range anywhere between 1 -22, I need to figure out a 
> way such that I need to update only those columns which are present
> 
> In the CSV file . This of course  I would like to accomplish using COPY 
> command , Please let me know if at all this is possible in postgresql 8.1
> 
> /Thanks and regards,/
> 
> Rajendra Singh
> 

In my opinion your best bet in terms of getting around the primary key 
violation is to create a temporary table without a primary key, copy 
your data into that table, then do a select into your main table from 
that table.

Eg.. I do the following:

CREATE TABLE creditors_temp_load AS SELECT * FROM creditors WHERE 1=0;
TRUNCATE TABLE creditors;
COPY creditors_temp_load FROM 'c:/temp/autodrs_creditors.txt' WITH 
DELIMITER AS '^' QUOTE '\f' CSV HEADER;
INSERT INTO creditors (SELECT DISTINCT ON (dealer_id,supplier_no) * FROM 
creditors_temp_load WHERE (dealer_id,supplier_no) is not null);

The first statement creates a copy of the 'creditors' table without any 
records (none in the creditors table have 1 equal to zero)
The second copies the data from the file into the temp table.
Finally an insert into the 'creditors' table is done by a select 
distinct on the temp table where the two fields listed are the primary 
key for that table.

I don't believe there is any way of getting around not having all the 
fields present - copy expects to find a match between fields in the file 
and fields in the destination table. If your record length in the load 
file is going to vary you may need to consider writing a program to read 
the data from the file and load it in.

Regards,
Paul.

-- 
Paul Lambert
Database Administrator
AutoLedgers

> In my opinion your best bet in terms of getting around the primary
> key 
> violation is to create a temporary table ...

good idea!  from my experience it's almost always best to pull raw info
into a buffer table before introducing it into the real world.


____________________________________________________________________________________
Bored stiff? Loosen up... 
Download and play hundreds of games for free on Yahoo! Games.
http://games.yahoo.com/games/front