Hi,
On 2023-02-18 15:51:06 +0530, Robert Haas wrote:
> On Thu, Feb 16, 2023 at 10:02 PM Andres Freund <andres@anarazel.de> wrote:
> > But there's nothing inherent in that. We know for certain which files we're
> > going to archive. And we don't need to work one-by-one. The archiver could
> > just start multiple subprocesses at the same time.
>
> But what if it doesn't want to start multiple processes, just
> multiplex within a single process?
To me that seems even simpler? Nothing but the archiver is supposed to create
.done files and nothing is supposed to remove .ready files without archiver
having created the .done files. So the archiver process can scan
archive_status until its done or until N archives have been collected, and
then process them at once? Only the creation of the .done files would be
serial, but I don't think that's commonly a problem (and could be optimized as
well, by creating multiple files and then fsyncing them in a second pass,
avoiding N filesystem journal flushes).
Maybe I am misunderstanding what you see as the problem?
Greetings,
Andres Freund