Re: [HACKERS] GSOC'17 project introduction: Parallel COPY executionwith errors handling

Поиск
Список
Период
Сортировка
От Alex K
Тема Re: [HACKERS] GSOC'17 project introduction: Parallel COPY executionwith errors handling
Дата
Msg-id CADfU8Wy3ZSSdkWpBsW-rkXR3OSg+LsWiGP3PEEN8dapiA7-x-w@mail.gmail.com
обсуждение исходный текст
Ответ на Re: [HACKERS] GSOC'17 project introduction: Parallel COPY executionwith errors handling  (Craig Ringer <craig@2ndquadrant.com>)
Ответы Re: [HACKERS] GSOC'17 project introduction: Parallel COPY executionwith errors handling  (Peter Geoghegan <pg@bowt.ie>)
Список pgsql-hackers
Дерево обсуждения
Hi pgsql-hackers,

Thank you again for all these replies. I have started working under this project 
and learnt a lot of new stuff last month, so here are some new thoughts about 
ERRORS handling in COPY. I decided to stick to the same thread, since it 
has a neutral subject.

(1) One of my mentors--Alvaro Herrera--suggested me to have a look on the 
UPSERT. It may be a good point to be able to achieve the same functionality 
as during the ON CONFLICT DO NOTHING, when COPY actually inserts tuples 
and errors handling is turned on. It could additionally reduce number of failed 
subtransactions and reduce XIDs consumption, while still ignoring some common 
errors like unique index violation.

Adding a full support of ON CONFLICT DO NOTHING/UPDATE to COPY seems 
to be a large separated task and is out of the current project scope, but maybe there is 
a relatively simple way to somehow perform internally tuples insert with 
ON CONFLICT DO NOTHING? I have added Peter Geoghegan to cc, as 
I understand he is the major contributor of UPSERT in PostgreSQL. It would be great
if he will answer this question.

(2) Otherwise, I am still going to use subtransactions via BeginInternalSubTransaction 
and PG_TRY / PG_CATCH with 
ReleaseCurrentSubTransaction / RollbackAndReleaseCurrentSubTransaction. 
To minimize XIDs consumption I will try to insert tuples in batches and pre-validate 
them as much as possible (as was suggested in the thread before).



Alexey



В списке pgsql-hackers по дате отправления:

Предыдущее
От: "David G. Johnston"
Дата:
Сообщение: Re: [HACKERS] PostgreSQL 10 changes in exclusion constraints - didsomething change? CASE WHEN behavior oddity
Следующее
От: Robert Haas
Дата:
Сообщение: Re: [HACKERS] statement_timeout is not working as expected with postgres_fdw