Re: [PATCH] Add archive_mode=follow_primary to prevent unarchived WAL on standby promotion

Hi,

On Fri, Oct 31, 2025 at 11:14 AM Andrey Borodin <x4mmm@yandex-team.ru> wrote:
>
> AFAIU archiver archives in order of reading archive_status directory, e.i. random order in worst case.
>

My understanding is the archiver uses a heap to allocate the batch of
files that will be archived to avoid scanning the directory
every-time. [0] The comparison is by name so it would only contain the
oldest WAL segments in order [1].

> Anyway, we could send .done signals to standby, but we cannot be sure given standby already have WAL for which we are
commandinghim to avoid archiving it... And standby might have these WALs from archive already, thus not needing .done
fileat all. 
>
> So, I implemented basic design that works for worst case. We can add some heuristics on top, but them must be
negligiblecheap in any possible archiving scenario. 
>

I was thinking at a high-level pgarch.c just has the latest WAL
segment archived from writer. Then every time before it attempts to
archive the segment in
pgarch_archiveXlog it just checks if the xlog is <
lastArchivedSegmentOnWriter. If it is earlier than the writer's
archived segment return true/skip the segment. It wouldn't matter if
the archived_segment on writer is ahead of what has been streamed to
the standby because standby archiver would only do comparisons against
what it has locally.

If writer has archived WAL 10, it should be safe for standby to skip WAL 1-9.
This way we don't need to stream every .done file from writer to
standby because we can rely on the fact that the segments are archived
in order.

[0] https://github.com/postgres/postgres/blob/master/src/backend/postmaster/pgarch.c#L739-L742
[1]  https://github.com/postgres/postgres/blob/master/src/backend/postmaster/pgarch.c#L792-L797

Thanks,
--
John Hsu - Amazon Web Services