Обсуждение: pgsql: Add support for MERGE SQL command

Add support for MERGE SQL command

MERGE performs actions that modify rows in the target table using a
source table or query. MERGE provides a single SQL statement that can
conditionally INSERT/UPDATE/DELETE rows -- a task that would otherwise
require multiple PL statements.  For example,

MERGE INTO target AS t
USING source AS s
ON t.tid = s.sid
WHEN MATCHED AND t.balance > s.delta THEN
  UPDATE SET balance = t.balance - s.delta
WHEN MATCHED THEN
  DELETE
WHEN NOT MATCHED AND s.delta > 0 THEN
  INSERT VALUES (s.sid, s.delta)
WHEN NOT MATCHED THEN
  DO NOTHING;

MERGE works with regular tables, partitioned tables and inheritance
hierarchies, including column and row security enforcement, as well as
support for row and statement triggers and transition tables therein.

MERGE is optimized for OLTP and is parameterizable, though also useful
for large scale ETL/ELT. MERGE is not intended to be used in preference
to existing single SQL commands for INSERT, UPDATE or DELETE since there
is some overhead.  MERGE can be used from PL/pgSQL.

MERGE does not support targetting updatable views or foreign tables, and
RETURNING clauses are not allowed either.  These limitations are likely
fixable with sufficient effort.  Rewrite rules are also not supported,
but it's not clear that we'd want to support them.

Author: Pavan Deolasee <pavan.deolasee@gmail.com>
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Author: Amit Langote <amitlangote09@gmail.com>
Author: Simon Riggs <simon.riggs@enterprisedb.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Reviewed-by: Andres Freund <andres@anarazel.de> (earlier versions)
Reviewed-by: Peter Geoghegan <pg@bowt.ie> (earlier versions)
Reviewed-by: Robert Haas <robertmhaas@gmail.com> (earlier versions)
Reviewed-by: Japin Li <japinli@hotmail.com>
Reviewed-by: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: Tomas Vondra <tomas.vondra@enterprisedb.com>
Reviewed-by: Zhihong Yu <zyu@yugabyte.com>
Discussion: https://postgr.es/m/CANP8+jKitBSrB7oTgT9CY2i1ObfOt36z0XMraQc+Xrz8QB0nXA@mail.gmail.com
Discussion: https://postgr.es/m/CAH2-WzkJdBuxj9PO=2QaO9-3h3xGbQPZ34kJH=HukRekwM-GZg@mail.gmail.com
Discussion: https://postgr.es/m/20201231134736.GA25392@alvherre.pgsql

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/7103ebb7aae8ab8076b7e85f335ceb8fe799097c

Modified Files
--------------
contrib/test_decoding/expected/ddl.out             |   46 +
contrib/test_decoding/sql/ddl.sql                  |   16 +
doc/src/sgml/libpq.sgml                            |    8 +-
doc/src/sgml/mvcc.sgml                             |   34 +-
doc/src/sgml/plpgsql.sgml                          |    5 +-
doc/src/sgml/ref/allfiles.sgml                     |    1 +
doc/src/sgml/ref/create_policy.sgml                |   23 +-
doc/src/sgml/ref/insert.sgml                       |   11 +-
doc/src/sgml/ref/merge.sgml                        |  620 +++++++
doc/src/sgml/reference.sgml                        |    1 +
doc/src/sgml/trigger.sgml                          |   22 +
src/backend/catalog/sql_features.txt               |    6 +-
src/backend/commands/explain.c                     |   35 +
src/backend/commands/trigger.c                     |  126 +-
src/backend/executor/README                        |   41 +-
src/backend/executor/execMain.c                    |   16 +
src/backend/executor/execPartition.c               |  113 +-
src/backend/executor/execReplication.c             |    2 +-
src/backend/executor/nodeModifyTable.c             |  936 +++++++++-
src/backend/executor/spi.c                         |    3 +
src/backend/nodes/copyfuncs.c                      |   55 +
src/backend/nodes/equalfuncs.c                     |   49 +
src/backend/nodes/nodeFuncs.c                      |   59 +-
src/backend/nodes/outfuncs.c                       |   36 +
src/backend/nodes/readfuncs.c                      |   43 +
src/backend/optimizer/plan/createplan.c            |   15 +-
src/backend/optimizer/plan/planner.c               |   64 +-
src/backend/optimizer/plan/setrefs.c               |   64 +-
src/backend/optimizer/prep/prepjointree.c          |   91 +
src/backend/optimizer/prep/preptlist.c             |   37 +
src/backend/optimizer/util/appendinfo.c            |   19 +-
src/backend/optimizer/util/pathnode.c              |   11 +-
src/backend/optimizer/util/plancat.c               |    4 +
src/backend/parser/Makefile                        |    1 +
src/backend/parser/analyze.c                       |   20 +-
src/backend/parser/gram.y                          |  180 +-
src/backend/parser/parse_agg.c                     |   10 +
src/backend/parser/parse_collate.c                 |    1 +
src/backend/parser/parse_expr.c                    |    4 +
src/backend/parser/parse_func.c                    |    3 +
src/backend/parser/parse_merge.c                   |  415 +++++
src/backend/parser/parse_relation.c                |   23 +-
src/backend/parser/parse_target.c                  |    3 +-
src/backend/rewrite/rewriteHandler.c               |   41 +-
src/backend/rewrite/rowsecurity.c                  |  106 +-
src/backend/tcop/pquery.c                          |    3 +
src/backend/tcop/utility.c                         |   16 +
src/backend/utils/adt/ruleutils.c                  |    9 +-
src/bin/psql/tab-complete.c                        |   69 +-
src/include/commands/trigger.h                     |    4 +-
src/include/executor/nodeModifyTable.h             |    3 +
src/include/executor/spi.h                         |    1 +
src/include/nodes/execnodes.h                      |   36 +-
src/include/nodes/nodes.h                          |    7 +-
src/include/nodes/parsenodes.h                     |   61 +-
src/include/nodes/pathnodes.h                      |    6 +-
src/include/nodes/plannodes.h                      |    7 +-
src/include/optimizer/pathnode.h                   |    2 +-
src/include/optimizer/prep.h                       |    1 +
src/include/parser/analyze.h                       |    5 +
src/include/parser/kwlist.h                        |    2 +
src/include/parser/parse_merge.h                   |   21 +
src/include/parser/parse_node.h                    |    5 +-
src/include/parser/parse_relation.h                |    3 +-
src/include/tcop/cmdtaglist.h                      |    1 +
src/interfaces/libpq/fe-exec.c                     |    9 +-
src/pl/plpgsql/src/pl_exec.c                       |    7 +-
src/pl/plpgsql/src/pl_gram.y                       |    8 +
src/pl/plpgsql/src/pl_unreserved_kwlist.h          |    1 +
src/pl/plpgsql/src/plpgsql.h                       |    2 +-
src/test/isolation/expected/merge-delete.out       |  117 ++
.../isolation/expected/merge-insert-update.out     |   94 +
.../isolation/expected/merge-match-recheck.out     |  116 ++
src/test/isolation/expected/merge-update.out       |  314 ++++
src/test/isolation/isolation_schedule              |    4 +
src/test/isolation/specs/merge-delete.spec         |   50 +
src/test/isolation/specs/merge-insert-update.spec  |   51 +
src/test/isolation/specs/merge-match-recheck.spec  |   77 +
src/test/isolation/specs/merge-update.spec         |  156 ++
src/test/regress/expected/identity.out             |   54 +
src/test/regress/expected/merge.out                | 1934 ++++++++++++++++++++
src/test/regress/expected/privileges.out           |   98 +
src/test/regress/expected/rowsecurity.out          |  182 ++
src/test/regress/expected/rules.out                |   32 +
src/test/regress/expected/triggers.out             |   48 +
src/test/regress/expected/with.out                 |  133 ++
src/test/regress/parallel_schedule                 |    2 +-
src/test/regress/sql/identity.sql                  |   46 +
src/test/regress/sql/merge.sql                     | 1273 +++++++++++++
src/test/regress/sql/privileges.sql                |  108 ++
src/test/regress/sql/rowsecurity.sql               |  156 ++
src/test/regress/sql/rules.sql                     |   33 +
src/test/regress/sql/triggers.sql                  |   47 +
src/test/regress/sql/with.sql                      |   56 +
src/tools/pgindent/typedefs.list                   |    4 +
95 files changed, 8726 insertions(+), 167 deletions(-)

Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
> Add support for MERGE SQL command

komodoensis just failed in a way that demonstrates instability of
the MERGE tests [1].  It looks to me like a background autovacuum
came along and processed one or both of ex_msource and ex_mtarget,
causing their default join plan to change to something else.
Probably an explicit ANALYZE would serve to stabilize that,
but it might end up with a slightly different plan than what
we have now.  It's not clear if that matters?

I also wonder whether we'd need to re-analyze after each of the
explain_merge calls, because those are changing the tables
and hence potentially provoking background stats updates.

It might be better to change these to temp tables.  You'd then
have to filter the temp schema out of the results; but since you
have a filtering function anyway, that doesn't seem too hard.

            regards, tom lane

[1] https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=komodoensis&dt=2022-09-19%2011%3A32%3A56

I wrote:
> It might be better to change these to temp tables.  You'd then
> have to filter the temp schema out of the results; but since you
> have a filtering function anyway, that doesn't seem too hard.

Or even easier, ALTER TABLE SET autovacuum_enabled = 'false'
before populating them.

            regards, tom lane

On 2022-Sep-19, Tom Lane wrote:

> I wrote:
> > It might be better to change these to temp tables.  You'd then
> > have to filter the temp schema out of the results; but since you
> > have a filtering function anyway, that doesn't seem too hard.
> 
> Or even easier, ALTER TABLE SET autovacuum_enabled = 'false'
> before populating them.

Thanks, I think I'll add WITH (autovacuum_enabled=false) to all the
CREATE TABLE statements there, as attached.  Tests running now, will
push shortly.

-- 
Álvaro Herrera        Breisgau, Deutschland  —  https://www.EnterpriseDB.com/
"Ed is the standard text editor."
      http://groups.google.com/group/alt.religion.emacs/msg/8d94ddab6a9b0ad3