Re: PITR promote bug: Checkpointer writes to older timeline

I wrote:
> Buildfarm member hornet just reported a failure in this test:
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hornet&dt=2021-06-27%2013%3A40%3A57
> It's not clear whether this is a problem with the test case or an
> actual server bug, but I'm leaning to the latter theory.  My gut
> feel is it's some problem in the "snapshot scalability" work.  It
> doesn't look the same as the known open issue, but maybe related?

Hmm, the plot thickens.  I scraped the buildfarm logs for similar-looking
assertion failures back to last August, when the snapshot scalability
patches went in.  The first such failure is not until 2021-03-24
(see attachment), and they all look to be triggered by
023_pitr_prepared_xact.pl.  It sure looks like recovering a prepared
transaction creates a transient state in which a new backend will
compute a broken snapshot.

            regards, tom lane

    sysname    | branch |      snapshot       |     stage     |
                                                     l
                                             

---------------+--------+---------------------+---------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 calliphoridae | HEAD   | 2021-03-24 06:50:09 | recoveryCheck | TRAP:
FailedAssertion("TransactionIdPrecedesOrEquals(TransactionXmin,RecentXmin)", File:
"/mnt/resource/andres/bf/calliphoridae/HEAD/pgsql.build/../pgsql/src/backend/storage/ipc/procarray.c",Line: 2463, PID:
3890215)
 francolin     | HEAD   | 2021-03-29 16:21:58 | recoveryCheck | TRAP:
FailedAssertion("TransactionIdPrecedesOrEquals(TransactionXmin,RecentXmin)", File:
"/mnt/resource/andres/bf/francolin/HEAD/pgsql.build/../pgsql/src/backend/storage/ipc/procarray.c",Line: 2463, PID:
1861665)
 moonjelly     | HEAD   | 2021-04-01 15:25:38 | recoveryCheck | TRAP:
FailedAssertion("TransactionIdPrecedesOrEquals(TransactionXmin,RecentXmin)", File: "procarray.c", Line: 2463, PID:
2345153)
 francolin     | HEAD   | 2021-04-07 12:30:08 | recoveryCheck | TRAP:
FailedAssertion("TransactionIdPrecedesOrEquals(TransactionXmin,RecentXmin)", File:
"/mnt/resource/andres/bf/francolin/HEAD/pgsql.build/../pgsql/src/backend/storage/ipc/procarray.c",Line: 2468, PID:
3257637)
 fairywren     | HEAD   | 2021-04-20 03:04:04 | recoveryCheck | TRAP:
FailedAssertion("TransactionIdPrecedesOrEquals(TransactionXmin,RecentXmin)", File:
"C:/tools/msys64/home/pgrunner/bf/root/HEAD/pgsql.build/../pgsql/src/backend/storage/ipc/procarray.c",Line: 2094, PID:
94824)
 mantid        | HEAD   | 2021-04-25 10:07:06 | recoveryCheck | TRAP:
FailedAssertion("TransactionIdPrecedesOrEquals(TransactionXmin,RecentXmin)", File: "procarray.c", Line: 2094, PID:
2820886)
 thorntail     | HEAD   | 2021-04-29 07:18:09 | recoveryCheck | TRAP:
FailedAssertion("TransactionIdPrecedesOrEquals(TransactionXmin,RecentXmin)", File:
"/home/nm/farm/sparc64_deb10_gcc_64_ubsan/HEAD/pgsql.build/../pgsql/src/backend/storage/ipc/procarray.c",Line: 2094,
PID:3099560) 
 mantid        | HEAD   | 2021-05-03 13:07:06 | recoveryCheck | TRAP:
FailedAssertion("TransactionIdPrecedesOrEquals(TransactionXmin,RecentXmin)", File: "procarray.c", Line: 2094, PID:
1163004)
 mantid        | HEAD   | 2021-05-10 01:07:07 | recoveryCheck | TRAP:
FailedAssertion("TransactionIdPrecedesOrEquals(TransactionXmin,RecentXmin)", File: "procarray.c", Line: 2468, PID:
2812704)
 hornet        | HEAD   | 2021-06-27 13:40:57 | recoveryCheck | TRAP:
FailedAssertion("TransactionIdPrecedesOrEquals(TransactionXmin,RecentXmin)", File: "procarray.c", Line: 2492, PID:
11862234)
(10 rows)