Re: Anti-critical-section assertion failure in mcxt.c reached by walsender

On Fri, May 07, 2021 at 01:18:19PM -0400, Tom Lane wrote:
> Realizing that 9989d37d prevents the assertion failure, I went
> to see if thorntail had shown EIO failures without assertions.
> Looking back 180 days, I found these:
> 
>   sysname  |    branch     |      snapshot       |       stage        |
                       l                                                                        
 
>
-----------+---------------+---------------------+--------------------+------------------------------------------------------------------------------------------------------------------------------------------------
>  thorntail | HEAD          | 2021-03-19 21:28:15 | recoveryCheck      | 2021-03-20 00:48:48.117 MSK [4089174:11]
008_fsm_truncation.plPANIC:  could not fdatasync file "000000010000000000000002": Input/output error
 
>  thorntail | HEAD          | 2021-04-06 16:08:10 | recoveryCheck      | 2021-04-06 19:30:54.103 MSK [3355008:11]
008_fsm_truncation.plPANIC:  could not fdatasync file "000000010000000000000002": Input/output error
 
>  thorntail | REL9_6_STABLE | 2021-04-12 02:38:04 | pg_basebackupCheck | pg_basebackup: could not fsync file
"000000010000000000000013":Input/output error
 
> 
> So indeed the kernel-or-hardware problem is affecting other branches.

Having a flaky buildfarm member is bad news.  I'll LD_PRELOAD the attached to
prevent fsync from reaching the kernel.  Hopefully, that will make the
hardware-or-kernel trouble unreachable.  (Changing 008_fsm_truncation.pl
wouldn't avoid this, because fsync=off doesn't affect syncs outside the
backend.)