Обсуждение: BUG #16842: pg_dump uses seek calls on pipe files: suggesting adding a flag to disable seek calls

The following bug has been logged on the website:

Bug reference:      16842
Logged by:          Tomas Dalebjörk
Email address:      tomas.dalebjork@gmail.com
PostgreSQL version: 13.0
Operating system:   SuSE 15
Description:

pg_dump is trying to identify the type of device used for output.
This is done in a way that it checks that filedescriptor to check if this
supports seek calls etc.
A redirect is a clone of the targeted filedescriptor, which will results in
that the pg_dump utility will enable seek calls.

https://searchcode.com/file/47798353/src/bin/pg_dump/pg_backup_custom.c/

It would be good to add a flag to disable this feature, as some devices does
not support seek calls, for example streaming devices such as a tape
drive.

From Red Hat:
# pg_dump > pgdump.bkp
"That's not a pipe; it's a redirect. The program's stdout (fd=1) will be for
that file, and they can still perform seek operations on it. An fstat(2) on
that file descriptor will show it as a regular file, not a character device
(tty) or a fifo (pipe)."
You could try this instead:
pg_dump | cat > /myfs/testfile.out"

Not sure if this is limited to when --format=custom is used?
It can be worth trying to use pg_dump without the --format=custom option
A restore from pg_dump can't be loaded using pg_restore if --format=custom
is left out (if I recall), but it can be loaded using "psql < dumpfile" 

Suggestions for how to avoid seek calls using pg_dump?
Is the seek calls only applicable when --format=custom is used?
Regards Tomas

PG Bug reporting form <noreply@postgresql.org> writes:
> It would be good to add a flag to disable this feature, as some devices does
> not support seek calls, for example streaming devices such as a tape
> drive.

That seems like an extremely niche use-case, so I'm unexcited about adding
a switch for it.  You can get the same results with a pipe, as your quoted
response from Red Hat support suggests.

Note that disabling seeking is not all that desirable in the first place,
as it will result (for -Fc output) in a dump file that is significantly
restrictive to restore performance, due to lack of data offset pointers.
You might really be best advised to rethink whether you want to write
directly to an unseekable device, rather than to a temp file you transfer
to tape later.

            regards, tom lane

thanks for the feedback Tom

is the seek enabled only if —format=custom
or is it enabled regardless of which flag used?

regards Tomas

Sent from my iPhone

> On 28 Jan 2021, at 16:32, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> PG Bug reporting form <noreply@postgresql.org> writes:
>> It would be good to add a flag to disable this feature, as some devices does
>> not support seek calls, for example streaming devices such as a tape
>> drive.
>
> That seems like an extremely niche use-case, so I'm unexcited about adding
> a switch for it.  You can get the same results with a pipe, as your quoted
> response from Red Hat support suggests.
>
> Note that disabling seeking is not all that desirable in the first place,
> as it will result (for -Fc output) in a dump file that is significantly
> restrictive to restore performance, due to lack of data offset pointers.
> You might really be best advised to rethink whether you want to write
> directly to an unseekable device, rather than to a temp file you transfer
> to tape later.
>
>            regards, tom lane

=?utf-8?Q?Tomas_Dalebj=C3=B6rk?= <tomas.dalebjork@gmail.com> writes:
> thanks for the feedback Tom
> is the seek enabled only if —format=custom
> or is it enabled regardless of which flag used?

There's no seeking required for plain-text output format, but of
course that's even less flexible.  Don't recall offhand about tar
format.

One thing I am kind of wondering about is why the whole thing is
a problem for you at all.  If the device refuses seeks, shouldn't
our probe fail, and then we'll go on our way without data offsets?

            regards, tom lane