Re: Add new COPY option REJECT_LIMIT

Поиск
Список
Период
Сортировка
От David G. Johnston
Тема Re: Add new COPY option REJECT_LIMIT
Дата
Msg-id CAKFQuwYP91_G6tktYFTZq_CmkZ_=zuWjkz1+25Nd8bpsrDkx5Q@mail.gmail.com
обсуждение исходный текст
Ответ на Add new COPY option REJECT_LIMIT  (torikoshia <torikoshia@oss.nttdata.com>)
Ответы Re: Add new COPY option REJECT_LIMIT  (torikoshia <torikoshia@oss.nttdata.com>)
Список pgsql-hackers
On Fri, Jan 26, 2024 at 2:49 AM torikoshia <torikoshia@oss.nttdata.com> wrote:
Hi,

9e2d870 enabled the COPY command to skip soft error, and I think we can
add another option which specifies the maximum tolerable number of soft
errors.

I remember this was discussed in [1], and feel it would be useful when
loading 'dirty' data but there is a limit to how dirty it can be.

Attached a patch for this.

What do you think?


I'm opposed to adding this particular feature.

When implementing this kind of business rule I'd need the option to specify a percentage, not just an absolute value.

I would focus on trying to put the data required to make this kind of determination into a place where applications implementing such business rules and monitoring can readily get at it.  The "ERRORS TO" and maybe a corresponding "STATS TO" option where a table can be specified for the system to place the problematic data and stats about the copy itself.

David J.

В списке pgsql-hackers по дате отправления:

Предыдущее
От: vignesh C
Дата:
Сообщение: Re: Memory consumed by child SpecialJoinInfo in partitionwise join planning
Следующее
От: Tom Lane
Дата:
Сообщение: Re: POC: GROUP BY optimization