Re: [PATCH] Initial progress reporting for COPY command

On 2020/06/22 17:21, Josef Šimánek wrote:
> 
> 
> po 22. 6. 2020 v 4:48 odesílatel Fujii Masao <masao.fujii@oss.nttdata.com <mailto:masao.fujii@oss.nttdata.com>>
napsal:
> 
> 
> 
>     On 2020/06/21 20:33, Josef Šimánek wrote:
>      >
>      >
>      > po 15. 6. 2020 v 6:39 odesílatel Fujii Masao <masao.fujii@oss.nttdata.com <mailto:masao.fujii@oss.nttdata.com>
<mailto:masao.fujii@oss.nttdata.com<mailto:masao.fujii@oss.nttdata.com>>> napsal:
 
>      >
>      >
>      >
>      >     On 2020/06/14 21:32, Josef Šimánek wrote:
>      >      > Hello, as proposed by Pavel Stěhule and discussed on local czech PostgreSQL maillist
(https://groups.google.com/d/msgid/postgresql-cz/CAFj8pRCZ42CBCa1bPHr7htffSV%2BNAcgcHHG0dVqOog4bsu2LFw%40mail.gmail.com?utm_medium=email&utm_source=footer),
Ihave prepared an initial patch for COPY command progress reporting.
 
>      >
>      >     Sounds nice!
>      >
>      >
>      >      > file - bool - is file is used?
>      >      > program - bool - is program used?
>      >
>      >     Are these fields really necessary in a progress view?
>      >     What values are reported when STDOUT/STDIN is specified in COPY command?
>      >
>      >
>      > For STDOUT and STDIN file is true and program is false.
> 
>     Could you tell me why these columns are necessary in *progress* view?
>     If we want to see what copy command is actually running, we can see
>     pg_stat_activity, instead. For example,
> 
>           SELECT pc.*, a.query FROM pg_stat_progress_copy pc, pg_stat_activity a WHERE pc.pid = a.pid;
> 
> If that doesn't make any sense, I can remove those. I have not strong opinion about those values. Those were just
aroundwhen I was looking for possible values to include in the progress report.
 

I vote not to expose them. *If* we expose them, we should also
expose the options in pg_stat_progress_xxx views, for example,
the options for BASE_BACKUP command in pg_stat_progress_basebackup,
for the consistency. But I don't think that makes sense.

> 
>      >
>      >      > file_bytes_processed - amount of bytes processed when file is used (otherwise 0), works for both
direction(
 
>      >      > FROM/TO) when file is used (file = t)
>      >
>      >     What value is reported when STDOUT/STDIN is specified in COPY command?
>      >
>      >
>      > For my first patch nothing was reported on STDOUT/STDIN usage. I'll attach new patch soon supporting those as
well.
> 
>     Thanks for the patch!
> 
>     With the patch, pg_stat_progress_copy seems to report the progress of
>     the processing on file_fdw. Is this intentional?
> 
> 
> Every action using internally COPY will be included in the progress report view.
> I have spotted for example pg_dump does that and is reported there as well.
> I do not see any problem regarding this. For pg_dump it is consistent with "pg_stat_activity" reporting COPY command
inthe query field.
 

So it's better to add this kind of information into the docs?

Regards,

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION