Fwd: [GENERAL] Does PG support bulk operation in embedded C

Поиск
Список
Период
Сортировка
От Steve Midgley
Тема Fwd: [GENERAL] Does PG support bulk operation in embedded C
Дата
Msg-id CAJexoS+hHw7NfL2btSDJ=W8gEf9Jr4tOy9jQStC1-tBV7Gtj5Q@mail.gmail.com
обсуждение исходный текст
Ответ на Re: [GENERAL] Does PG support bulk operation in embedded C  (Steve Midgley <steve.midgley@learningtapestry.com>)
Список pgsql-sql
<div dir="ltr"><div class="gmail_quote"><p dir="ltr">You can do in-memory transfers using 'copy' - you do that with the
stdoutoption. Performance is very good if you're on the same machine as the sql server. As far as I could see the
transfernever touched the disk (once I loaded the data into memory from a file) and I was getting blistering insert
speeds(this was back in 2009 but if I remember right, the copy speed using this method was 50x the speed of a
traditionalinsert command, even when a bunch of inserts were wrapped in a transaction).<br /><br />I dug up the code I
usedback then - it's in Ruby, using ActiveRecord, and is from 2009 (hasn't been used since). But maybe it is useful for
portingto your environment: <a
href="https://gist.github.com/science/15e97e414d5666c2f486">https://gist.github.com/science/15e97e414d5666c2f486</a><br
/><br/>Obviously network is a likely bottleneck if you're not on the same box.<p dir="ltr"><span
style="color:rgb(80,0,80)">OnMay 20, 2015 8:24 AM, "Ravi Krishna" <</span><a href="mailto:srkrishna@gmx.com"
target="_blank">srkrishna@gmx.com</a><spanstyle="color:rgb(80,0,80)">> wrote:</span><br /><div class=""><div
class="h5"><divclass="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><br/> >
Anyway,taking the function name 'bulk' into account - I think you want<br /> > find the best way to to insert a
largenumber of row/records.<br /> ><br /> > Review the 'copy' command to populate a database:<br /> > <a
href="http://www.postgresql.org/docs/9.1/static/populate.html"
target="_blank">http://www.postgresql.org/docs/9.1/static/populate.html</a> - read<br /> > carefully as it will
increaseperformance in a dramatic way.<br /><br /> I am aware of the copy command, but we are talking about application
insertingrows from a C++ task<br /> with values coming in variables. Does COPY handle that? COPY looks more closer to
sqlloaderof Oracle.<br /><br /> DB2 Bulkcopy API is damn good. We have applications inserting tens of rows in one
singlecall.<br /><br /><br /> --<br /> Sent via pgsql-sql mailing list (<a href="mailto:pgsql-sql@postgresql.org"
target="_blank">pgsql-sql@postgresql.org</a>)<br/> To make changes to your subscription:<br /><a
href="http://www.postgresql.org/mailpref/pgsql-sql"target="_blank">http://www.postgresql.org/mailpref/pgsql-sql</a><br
/></blockquote></div></div></div></div><br/></div> 

В списке pgsql-sql по дате отправления:

Предыдущее
От: Steve Midgley
Дата:
Сообщение: Re: [GENERAL] Does PG support bulk operation in embedded C
Следующее
От: Suresh Raja
Дата:
Сообщение: extracting PII data and transforming it across table.