Обсуждение: BUG #15929: logical decoding can not write down the analyse result when the output file is touched.

The following bug has been logged on the website:

Bug reference:      15929
Logged by:          movead li
Email address:      movead.li@highgo.ca
PostgreSQL version: 12beta1
Operating system:   centos7
Description:

I was doing some testing of logical decoding and ran into a scenario that
may be a bug. The following steps will lead to reproducing the behaviour :


1.Start a logical decoding use below command:

pg_recvlogical -d postgres --slot=test2 --create-slot
pg_recvlogical -d postgres --slot=test2 --start -f ld.log

2. When I do a DML on the database it writes down the info into ld.log
file,
so far so good.

3. Next I open the log file. “Id.log” with vi and make some arbitrary
changes
in the log file.

4. If I run a DML command now the DML info will not appear in the ld.log
file. 
Refer to step 2, when I did a DML it was appearing in the log file.

5. If I do some manipulation with the log file using the OS commands below :


mv ld.log ld.log1; 
cp ld.log1 ld.log;

Then the DML info will start appearing in file ld.log1.

This seems like a bug to be because the user might accidentally some data
without noticing.

I have looked at the code to debug this behaviour and find a solution

In file ‘pg_recvlogical.c’ the ‘StreamLogicalLog()’ function seem to have
a
issue with handling of DML info. It uses open() to open the file and it does
not
check the file state when writing to the file after a long wait.

Hi,

On 2019-07-27 09:39:47 +0000, PG Bug reporting form wrote:
> I was doing some testing of logical decoding and ran into a scenario that
> may be a bug. The following steps will lead to reproducing the behaviour :
> 
> 
> 1.Start a logical decoding use below command:
> 
> pg_recvlogical -d postgres --slot=test2 --create-slot
> pg_recvlogical -d postgres --slot=test2 --start -f ld.log
> 
> 2. When I do a DML on the database it writes down the info into ld.log
> file,
> so far so good.
> 
> 3. Next I open the log file. “Id.log” with vi and make some arbitrary
> changes
> in the log file.
> 
> 4. If I run a DML command now the DML info will not appear in the ld.log
> file. 
> Refer to step 2, when I did a DML it was appearing in the log file.
> 
> 5. If I do some manipulation with the log file using the OS commands below :
> 
> 
> mv ld.log ld.log1; 
> cp ld.log1 ld.log;
> 
> Then the DML info will start appearing in file ld.log1.
> 
> This seems like a bug to be because the user might accidentally some data
> without noticing.
> 
> I have looked at the code to debug this behaviour and find a solution
> 
> In file ‘pg_recvlogical.c’ the ‘StreamLogicalLog()’ function seem to have
> a
> issue with handling of DML info. It uses open() to open the file and it does
> not
> check the file state when writing to the file after a long wait.

I don't see a bug here. It's completely normal for a tool that writes to
a file to continue writing into the file it started to write to. The
reason that vi breaks that is presumably just that it renames the
"edited" version into the place of the file that pg_recvlogical is still
writing to.

Greetings,

Andres Freund

 

>I don't see a bug here. It's completely normal for a tool that writes to

>a file to continue writing into the file it started to write to. The

>reason that vi breaks that is presumably just that it renames the

>"edited" version into the place of the file that pg_recvlogical is still

>writing to.

It is the issue that the target file is changed but the 'pg_recvlogical'

process doesn't know it at all. So it streams all wal changes but writes 
down nothing at last.

So I think we can recreate or reopen the changed file when there be
something wrong with the target file. And we can choose to kill the 
'pg_recvlogical' process because it's no use at all now, or we should
give a warning to users at lease.

---

Highgo Software (Canada/China/Pakistan) 

URL : www.highgo.ca 

EMAIL: mailto:movead(dot)li(at)highgo(dot)ca