On Thu, Jan 25, 2024 at 11:25 AM torikoshia <torikoshia@oss.nttdata.com> wrote:
>
> On 2024-01-24 17:05, Masahiko Sawada wrote:
> > On Tue, Jan 23, 2024 at 1:02 AM torikoshia <torikoshia@oss.nttdata.com>
> > wrote:
> >>
> >> On 2024-01-17 14:47, Masahiko Sawada wrote:
> >> > On Wed, Jan 17, 2024 at 2:22 PM torikoshia <torikoshia@oss.nttdata.com>
> >> > wrote:
> >> >>
> >> >> Hi,
> >> >>
> >> >> 132de9968840c introduced SAVE_ERROR_TO option to COPY and enabled to
> >> >> skip malformed data, but there is no way to watch the number of
> >> >> skipped
> >> >> rows during COPY.
> >> >>
> >> >> Attached patch adds tuples_skipped to pg_stat_progress_copy, which
> >> >> counts the number of skipped tuples because source data is malformed.
> >> >> If SAVE_ERROR_TO is not specified, this column remains zero.
> >> >>
> >> >> The advantage would be that users can quickly notice and stop COPYing
> >> >> when there is a larger amount of skipped data than expected, for
> >> >> example.
> >> >>
> >> >> As described in commit log, it is expected to add more choices for
> >> >> SAVE_ERROR_TO like 'log' and using such options may enable us to know
> >> >> the number of skipped tuples during COPY, but exposed in
> >> >> pg_stat_progress_copy would be easier to monitor.
> >> >>
> >> >>
> >> >> What do you think?
> >> >
> >> > +1
> >> >
> >> > The patch is pretty simple. Here is a comment:
> >> >
> >> > + (if <literal>SAVE_ERROR_TO</literal> is specified, otherwise
> >> > zero).
> >> > + </para></entry>
> >> > + </row>
> >> >
> >> > To be precise, this counter only advances when a value other than
> >> > 'ERROR' is specified to SAVE_ERROR_TO option.
> >>
> >> Thanks for your comment and review!
> >>
> >> Updated the patch according to your comment and option name change by
> >> b725b7eec.
> >
> > Thanks! The patch looks good to me. I'm going to push it tomorrow,
> > barring any objections.
>
> Thanks!
Pushed (commit 729439607).
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com