pgsql: read_stream: Introduce and use optional batchmode support

read_stream: Introduce and use optional batchmode support

Submitting IO in larger batches can be more efficient than doing so
one-by-one, particularly for many small reads. It does, however, require
the ReadStreamBlockNumberCB callback to abide by the restrictions of AIO
batching (c.f. pgaio_enter_batchmode()). Basically, the callback may not:
a) block without first calling pgaio_submit_staged(), unless a
   to-be-waited-on lock cannot be part of a deadlock, e.g. because it is
   never held while waiting for IO.

b) directly or indirectly start another batch pgaio_enter_batchmode()

As this requires care and is nontrivial in some cases, batching is only
used with explicit opt-in.

This patch adds an explicit flag (READ_STREAM_USE_BATCHING) to read_stream and
uses it where appropriate.

There are two cases where batching would likely be beneficial, but where we
aren't using it yet:

1) bitmap heap scans, because the callback reads the VM

   This should soon be solved, because we are planning to remove the use of
   the VM, due to that not being sound.

2) The first phase of heap vacuum

   This could be made to support batchmode, but would require some care.

Reviewed-by: Noah Misch <noah@leadboat.com>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/uvrtrknj4kdytuboidbhwclo4gxhswwcpgadptsjvjqcluzmah%40brqs62irg4dt

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/ae3df4b34155d4a3811762e78d5076e64df66fba

Modified Files
--------------
contrib/amcheck/verify_heapam.c       | 13 ++++++++++++-
contrib/pg_prewarm/pg_prewarm.c       |  7 ++++++-
contrib/pg_visibility/pg_visibility.c |  8 +++++++-
src/backend/access/gist/gistvacuum.c  |  8 +++++++-
src/backend/access/heap/heapam.c      | 16 +++++++++++++++-
src/backend/access/heap/vacuumlazy.c  | 21 ++++++++++++++++++---
src/backend/access/nbtree/nbtree.c    |  8 +++++++-
src/backend/access/spgist/spgvacuum.c |  8 +++++++-
src/backend/commands/analyze.c        |  7 ++++++-
src/backend/storage/aio/read_stream.c | 16 ++++++++++++++++
src/backend/storage/buffer/bufmgr.c   |  8 +++++++-
src/include/storage/read_stream.h     | 21 +++++++++++++++++++++
12 files changed, 129 insertions(+), 12 deletions(-)