> po 15. 6. 2020 v 7:34 odesílatel Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> napsal:
>>
>> > I'm using ftell to get current position in file to populate file_bytes_processed without error handling (ftell can
return-1L and also populate errno on problems).
>> >
>> > 1. Is that a good way to get progress of file processing?
>>
>> IMO, it's better to handle the error cases. One possible case where
>> ftell can return -1 and set errno is when the total bytes processed is
>> more than LONG_MAX.
>>
>> Will your patch handle file_bytes_processed reporting for COPY FROM
>> STDIN cases? For this case, ftell can't be used.
>>
>> Instead of using ftell and worrying about the errors, a simple
>> approach could be to have a uint64 variable in CopyStateData to track
>> the number of bytes read whenever CopyGetData is called. This approach
>> can also handle the case of COPY FROM STDIN.
>
>
> Thanks for suggestion. I used this approach and latest patch supports both STDIN and STDOUT now.
>
Thanks.
It would be good to see the performance of the copy command(probably
with a few GBs of data) with patch and without patch for both csv/text
and binary files.
For copy from command CopyGetData gets called for every
RAW_BUF_SIZE(64KB) and so is CopyUpdateBytesProgress function, but for
binary format files, CopyGetData gets called for each field/column for
all rows/lines/tuples.
Can we make CopyUpdateBytesProgress() a macro or an inline
function(probably by using pg_attribute_always_inline) to reduce
function call overhead as it just handles two statements?
I tried to apply the patch on commit #
7ce461560159948ba0c802c767e42c5f5ae08b4a, seems like a warning.
bharath:postgres$ git apply /mnt/hgfs/Downloads/copy-progress-v2.diff
/mnt/hgfs/Downloads/copy-progress-v2.diff:277: trailing whitespace.
* for counting tuples inserted by an INSERT
command. Update
warning: 1 line adds whitespace errors.
With Regards,
Bharath Rupireddy.
EnterpriseDB: http://www.enterprisedb.com