On Tue, Dec 8, 2020 at 3:56 PM Craig Ringer
<craig.ringer@enterprisedb.com> wrote:
> I thought I'd start the discussion on this and see where we can go with it. What incremental steps can be done to
moveus toward parallelisable I/O without having to redesign everything?
>
> I'm thinking that redo is probably a good first candidate. It doesn't depend on the guts of the executor. It is much
lesssensitive to ordering between operations in shmem and on disk since it runs in the startup process. And it hurts
REALLYBADLY from its single-threaded blocking approach to I/O - as shown by an extension written by 2ndQuadrant that
candouble redo performance by doing read-ahead on btree pages that will soon be needed.
About the redo suggestion: https://commitfest.postgresql.org/31/2410/
does exactly that! It currently uses POSIX_FADV_WILLNEED because
that's what PrefetchSharedBuffer() does, but when combined with a
"real AIO" patch set (see earlier threads and conference talks on this
by Andres) and a few small tweaks to control batching of I/O
submissions, it does exactly what you're describing. I tried to keep
the WAL prefetcher project entirely disentangled from the core AIO
work, though, hence the "poor man's AIO" for now.