Re: Best COPY Performance

Поиск
Список
Период
Сортировка
От Markus Schaber
Тема Re: Best COPY Performance
Дата
Msg-id 453C8B0D.4070803@logix-tt.com
обсуждение исходный текст
Ответ на Best COPY Performance  ("Worky Workerson" <worky.workerson@gmail.com>)
Ответы Re: Best COPY Performance  ("Luke Lonergan" <llonergan@greenplum.com>)
Re: Best COPY Performance  ("Worky Workerson" <worky.workerson@gmail.com>)
Список pgsql-performance
Hi, Worky,

Worky Workerson wrote:
> I am currently getting
> between 10K and 15K inserts/second.

> I ran trivial little insert into a table with a single integer row and
> came close to 250K inserts/second using psql's \copy, so I'm thinking
> that my code could be optimized a bit more, but wanted to check around
> to see if that was the case.

Could you COPY one of your tables out to disk via psql, and then COPY it
back into the database, to reproduce this measurement with your real data?

Also, how much is the disk load, and CPU usage?

As long as psql is factor 20 better than your perl script, I think that
the perl interface is what should be optimized.

On a table with no indices, triggers and contstraints, we managed to
COPY about 7-8 megabytes/second with psql over our 100 MBit network, so
here the network was the bottleneck.

You should think about making your perl program writing the COPY
statement as text, and piping it into psql.

HTH,
Markus
--
Markus Schaber | Logical Tracking&Tracing International AG
Dipl. Inf.     | Software Development GIS

Fight against software patents in Europe! www.ffii.org
www.nosoftwarepatents.org

В списке pgsql-performance по дате отправления:

Предыдущее
От: Richard Huxton
Дата:
Сообщение: Re: Vacuum and Memory Loss
Следующее
От: John Philips
Дата:
Сообщение: Re: Optimizing disk throughput on quad Opteron