Re: Loading lots of data in a SQL command

Поиск
Список
Период
Сортировка
От Richard Huxton
Тема Re: Loading lots of data in a SQL command
Дата
Msg-id 43BB9392.5080805@archonet.com
обсуждение исходный текст
Ответ на Loading lots of data in a SQL command  (frank church <pgsql@adontendev.net>)
Список pgsql-sql
frank church wrote:
> I am load lots of data via SQL into a database and wrapping it into transactions
> speeds it up.
> 
> However this fails a number of times. The queries results are logged so it is
> easy for me to find problem records.
> 
> However a single failure causes the whole transaction to fail.>
> Is there a setting or feature that allows which allows the same performance as
> transactions, without causing the whole process to fail, like a delayed updates
> or write mechanism of some sort.

Not as it stands. I tend to use a small perl wrapper myself that loads 
in batches of e.g. 10000 rows and if there is an error deal with it 
separately.

I seem to recall it being discussed as a built-in feature recently 
though, so there might be someone working on it for a future version.

> It is something I would like to set in that particular data looad.

You might find the "pgloader" project meets your needs exactly:  http://pgfoundry.org/projects/pgloader/

--   Richard Huxton  Archonet Ltd


В списке pgsql-sql по дате отправления:

Предыдущее
От: Michael Fuhr
Дата:
Сообщение: Re: getting a query column to return 0 if nothing matches
Следующее
От: Mario Splivalo
Дата:
Сообщение: Regular Expression Matching problem...