pgsql: Support asynchronous execution of MergeAppend subplans.
pgsql: Support asynchronous execution of MergeAppend subplans.
От:
Etsuro Fujita <efujita@postgresql.org>
Дата:
Support asynchronous execution of MergeAppend subplans. Commit 27e1f1456 added asynchronous execution for Append nodes, letting multiple async-capable children -- in practice postgres_fdw ForeignScans -- run concurrently instead of one at a time, but MergeAppend was not covered. This commit extends asynchronous execution to MergeAppend. On the first call, requests for all valid async subplans are fired off up front, the synchronous subplans are pulled while those requests are in flight, and the async results are then collected before the merge heap is built. After that, the merge cannot emit another row until the next tuple from one specific subplan (the current heap top) is known, so at most one request is outstanding during steady-state execution, and we always wait indefinitely for it rather than polling as Append does. Some other notable changes to the core and postgres_fdw code: * create_merge_append_plan now marks children async-capable. Unlike a plain Append, which disables async whenever its output must be ordered, a MergeAppend always has pathkeys and still preserves ordering by merging the per-child streams, so having pathkeys is not disqualifying here; a child is eligible only if it is ordered well enough on its own that no Sort node was inserted above it, as Sort is not async-capable. The new GUC enable_async_merge_append (default on) controls this. * The event-loop body of ExecAppendAsyncEventWait is factored out into a new ExecAsyncEventWait in execAsync.c and shared by both node types, with the caller responsible for creating and freeing the WaitEventSet. A new MergeAppendReady wait event reports waits in the new event loop. * Since an FDW's ForeignAsyncConfigureWait callback can no longer assume the requestor is an Append, add ExecAsyncGetRequestorEventSet to fetch the requestor's WaitEventSet without knowing its node type, and use it in postgres_fdw. * postgresForeignAsyncConfigureWait also needs adjusting: the optimization of skipping event configuration, rather than processing an in-progress request made by a different requestor, is valid only when our requestor is an Append; a MergeAppend cannot make progress unless the given request produces a result, so in that case we process the pending request and configure the event. Full disclosure: I (Etsuro) reworked Alexander Pyhalov's original patch with Claude Opus 4.8. Author: Alexander Pyhalov Author: Etsuro Fujita Reviewed-by: Matheus Alcantara Reviewed-by: Alexander Korotkov Reviewed-by: Alena Rybakina Reviewed-by: Álvaro Herrera Reviewed-by: solai v Reviewed-by: Alexander Pyhalov Reviewed-by: Etsuro Fujita Discussion: https://postgr.es/m/59be194c5a409fb9fc9f2031581b8a44%40postgrespro.ru Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/11591f273e6a26fb55728b410538fa2627b19b28 Modified Files -------------- contrib/postgres_fdw/expected/postgres_fdw.out | 442 +++++++++++++++++++++++- contrib/postgres_fdw/postgres_fdw.c | 60 ++-- contrib/postgres_fdw/sql/postgres_fdw.sql | 119 ++++++- doc/src/sgml/config.sgml | 14 + doc/src/sgml/fdwhandler.sgml | 24 +- doc/src/sgml/postgres-fdw.sgml | 10 +- src/backend/executor/README | 15 +- src/backend/executor/execAsync.c | 127 +++++++ src/backend/executor/nodeAppend.c | 94 +---- src/backend/executor/nodeMergeAppend.c | 301 +++++++++++++++- src/backend/optimizer/path/costsize.c | 1 + src/backend/optimizer/plan/createplan.c | 26 ++ src/backend/utils/activity/wait_event_names.txt | 1 + src/backend/utils/misc/guc_parameters.dat | 7 + src/backend/utils/misc/postgresql.conf.sample | 1 + src/include/executor/execAsync.h | 8 + src/include/executor/nodeMergeAppend.h | 1 + src/include/nodes/execnodes.h | 6 + src/include/optimizer/cost.h | 1 + src/test/regress/expected/sysviews.out | 3 +- 20 files changed, 1115 insertions(+), 146 deletions(-)