Re: Using COPY to import large xml file

On 06/24/2018 06:07 PM, Anto Aravinth wrote:
> Thanks for the response. I'm not sure, how long does this tool takes for the
> 70GB data. 

In my memory, it took several hours. I can't remember if it is xml conversion or
insert which are longer.

> 
> I used node to stream the xml files into inserts.. which was very slow..
> Actually the xml contains 40 million records, out of which 10Million took around
> 2 hrs using nodejs. Hence, I thought will use COPY command, as suggested on the
> internet. 
> 
> Definitely, will try the code and let you know.. But looks like it uses the same
> INSERT, not copy.. interesting if it runs quick on my machine. 

Yes it use INSERT, maybe it is not difficult to change the code to use COPY instead.

-- 
Adrien NAYRAT
https://blog.anayrat.info