[ANNOUNCE] == PostgreSQL Weekly News - September 04 2017 ==

== PostgreSQL Weekly News - September 04 2017 ==

PostgreSQL 10 Beta 4 released.  Test!
https://www.postgresql.org/about/news/1776/

PostgreSQL 9.6.5, 9.5.9, 9.4.14, 9.3.19, and 9.2.23 released. Upgrade ASAP.
https://www.postgresql.org/about/news/1777/

PostgreSQL Session November 17th, 2017, in Paris, France.  The CfP is open until
September 30, 2017 at call-for-paper AT postgresql-sessions DOT org.  Please
include any of the following which apply: name, surname, twitter handle,
company, short biography (contributions to the PostgreSQL community), talk
title, talk abstract, and any special needs.

== PostgreSQL Product News ==

Ora2Pg 18.2, a tool for migrating Oracle databases to PostgreSQL, released.
http://ora2pg.darold.net/

PostGIS 2.4.0 beta1, the industry standard geographic information system package
for PostgreSQL, released.
http://postgis.net/2017/09/02/postgis-2.4.0beta1/

Slony 2.2.6, a trigger-based logical replication system for PostgreSQL,
released.
http://www.slony.info/

pgFormatter 2.1, a formatter/beautifier for SQL code, released.
http://sqlformat.darold.net/

Database Designer for PostgreSQL 1.12-beta released.
http://microolap.com/products/database/postgresql-designer/

== PostgreSQL Jobs for September ==

http://archives.postgresql.org/pgsql-jobs/2017-09/

== PostgreSQL Local ==

PostgresOpen will take place September 6-8, 2017 in San Francisco.
https://2017.postgresopen.org/

PGBR2017 will take place in Porto Alegre, Rio Grande do Sul,
Brazil on September 14-16 2017.
https://pgbr.postgresql.org.br/2017/

PGDay.IT 2017 will take place October 13th, in Milan, Italy.
http://pgday.it

PostgreSQL Conference Europe 2017 will be held on October 24-27, 2017 in the
Warsaw Marriott Hotel, in Warsaw, Poland.
https://2017.pgconf.eu/

PGConf Local: Seattle will be held November 13 - 14, 2017.
https://www.pgconf.us/#Seattle2017

PGDay Australia 2017 will be held on November 17 in Melbourne.  The CfP is open
through September 8, 2017 at https://goo.gl/5d9EbT
http://2017.pgday.com.au/

2ndQuadrant PostgreSQL Conference 2017 (2Q PGConf, for short) will be hosted on
November 6th & 7th in New York City, and November 9th in Chicago.
http://www.2qpgconf.com/

PGConf Local: Austin will be held December 4 - 5, 2017. Call for Papers is
now open at https://www.pgconf.us/conferences/Austin2017

PGConf.ASIA 2017 will take place on December 4-6 2017 in Akihabara, Tokyo,
Japan.
http://www.pgconf.asia/EN/2017/

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david@fetter.org, German language
to pwn@pgug.de, Italian language to pwn@itpug.org.

== Applied Patches ==

Peter Eisentraut pushed:

- Clarify documentation.  Discussion:
  https://www.postgresql.org/message-id/flat/20170618071607.GA16418%40nol.local
  https://git.postgresql.org/pg/commitdiff/46596f8d6b5dfdb836fa79788614a3786bc3cbb6

- doc: Avoid sidebar element.  The formatting of the sidebar element didn't
  carry over to the new tool chain.  Instead of inventing a whole new way of
  dealing with it, just convert the one use to a "note".
  https://git.postgresql.org/pg/commitdiff/00f6d5c2c3ae2f6d198e41800e5edcf0150d485b

- Remove to pre-8.2 coding convention for PG_MODULE_MAGIC.  PG_MODULE_MAGIC has
  been around since 8.2, with 8.1 long since in EOL, so remove the mention of
  #ifdef guards for compiling against pre-8.2 sources from the documentation.
  Author: Daniel Gustafsson <daniel@yesql.se>
  https://git.postgresql.org/pg/commitdiff/b5c75feca7ffb2667c42b86286e262d6cb709b76

- doc: Remove mentions of server-side CRL and CA file names.  Commit
  a445cb92ef5b3a31313ebce30e18cc1d6e0bdecb removed the default file names for
  server-side CRL and CA files, but left them in the docs with a small note.
  This removes the note and the previous default names to clarify, as well as
  changes mentions of the file names to make it clearer that they are
  configurable.  Author: Daniel Gustafsson <daniel@yesql.se> Reviewed-by:
  Michael Paquier <michael.paquier@gmail.com>
  https://git.postgresql.org/pg/commitdiff/a0572203532560423c92066b90d13383720dce3a

- Add memory info to getrusage output.  Add the maxrss field to the getrusage
  output (log_*_stats).  This was previously omitted because of portability
  concerns, but we feel this might not be a concern anymore.  based on patch by
  Justin Pryzby <pryzby@telsasoft.com>
  https://git.postgresql.org/pg/commitdiff/c039ba0716383ccaf88c9be1a7f0803a77823de1

- doc: Fix typos and other minor issues.  Author: Alexander Lakhin
  <exclusion@gmail.com>
  https://git.postgresql.org/pg/commitdiff/afc58affb6616a415ea991763e0383832346e7c7

Tom Lane pushed:

- Fix over-aggressive sanity check in misc_sanity.sql.  Fix thinko in commit
  8be8510cf: it's okay to have dbid == 0 in normal (non-pin) entries in
  pg_shdepend, because global objects such as databases are entered that way.
  The test would pass so long as it was run in a cluster containing no
  databases/tablespaces owned by, or granted to, roles other than the bootstrap
  superuser.  That's the expected situation for "make check", but for "make
  installcheck", not so much.  Reported by Ryan Murphy.  Discussion:
  https://postgr.es/m/CAHeEsBc6EQe0mxGBKDXAwJbntgfvoAd5MQC-5362SmC3Tng_6g@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/95e28b7f0c9e30cb636f58b5aa351e5f8ae3a473

- Doc: adjust release-note credit for parallel pg_restore fix.  Discussion:
  https://postgr.es/m/CAFcNs+pJ6_Ud-zg3vY_Y0mzfESdM34Humt8avKrAKq_H+v18Cg@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/dc4f356b4470bcdcaf1d0fb68444683206eeba22

- Release notes for 9.6.5, 9.5.9, 9.4.14, 9.3.19, 9.2.23.
  https://git.postgresql.org/pg/commitdiff/f97c55c708a6fbcb675be6fbb94af3038733dfa4

- Improve docs about numeric formatting patterns (to_char/to_number).  The
  explanation about "0" versus "9" format characters was confusing and arguably
  wrong; the discussion of sign handling wasn't very good either.  Notably,
  while it's accurate to say that "FM" strips leading zeroes in date/time
  values, what it really does with numeric values is to strip *trailing* zeroes,
  and then only if you wrote "9" rather than "0".  Per gripes from Erwin
  Brandstetter.  Discussion:
  https://postgr.es/m/CAGHENJ7jgRbTn6nf48xNZ=FHgL2WQ4X8mYsUAU57f-vq8PubEw@mail.gmail.com
  Discussion:
  https://postgr.es/m/CAGHENJ45ymd=GOCu1vwV9u7GmCR80_5tW0fP9C_gJKbruGMHvQ@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/ce5dcf54b942a469194ae390730f803b3f3fb928

- Teach libpq to detect integer overflow in the row count of a PGresult.  Adding
  more than 1 billion rows to a PGresult would overflow its ntups and tupArrSize
  fields, leading to client crashes.  It'd be desirable to use wider fields on
  64-bit machines, but because all of libpq's external APIs use plain "int" for
  row counters, that's going to be hard to accomplish without an ABI break.
  Given the lack of complaints so far, and the general pain that would be
  involved in using such huge PGresults, let's settle for just preventing the
  overflow and reporting a useful error message if it does happen.  Also, for a
  couple more lines of code we can increase the threshold of trouble from
  INT_MAX/2 to INT_MAX rows.  To do that, refactor pqAddTuple() to allow
  returning an error message that replaces the default assumption that it failed
  because of out-of-memory.  Along the way, fix PQsetvalue() so that it reports
  all failures via pqInternalNotice().  It already did so in the case of bad
  field number, but neglected to report anything for other error causes.
  Because of the potential for crashes, this seems like a back-patchable bug
  fix, despite the lack of field reports.  Michael Paquier, per a complaint from
  Igor Korot.  Discussion:
  https://postgr.es/m/CA+FnnTxyLWyjY1goewmJNxC==HQCCF4fKkoCTa9qR36oRAHDPw@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/2e70d6b5e99b7e7b53336b1838f869bbea1b5024

- Doc: document libpq's restriction to INT_MAX rows in a PGresult.  As long as
  PQntuples, PQgetvalue, etc, use "int" for row numbers, we're pretty much stuck
  with this limitation.  The documentation formerly stated that the result of
  PQntuples "might overflow on 32-bit operating systems", which is just
  nonsense: that's not where the overflow would happen, and if you did reach an
  overflow it would not be on a 32-bit machine, because you'd have OOM'd long
  since.  Discussion:
  https://postgr.es/m/CA+FnnTxyLWyjY1goewmJNxC==HQCCF4fKkoCTa9qR36oRAHDPw@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/6cbee65eee20c144bb4de169c6802f20e76785b0

- Force rescanning of parallel-aware scan nodes below a Gather[Merge].  The
  ExecReScan machinery contains various optimizations for postponing or skipping
  rescans of plan subtrees; for example a HashAgg node may conclude that it can
  re-use the table it built before, instead of re-reading its input subtree.
  But that is wrong if the input contains a parallel-aware table scan node,
  since the portion of the table scanned by the leader process is likely to vary
  from one rescan to the next.  This explains the timing-dependent buildfarm
  failures we saw after commit a2b70c89c.  The established mechanism for showing
  that a plan node's output is potentially variable is to mark it as depending
  on some runtime Param.  Hence, to fix this, invent a dummy Param (one that has
  a PARAM_EXEC parameter number, but carries no actual value) associated with
  each Gather or GatherMerge node, mark parallel-aware nodes below that node as
  dependent on that Param, and arrange for ExecReScanGather[Merge] to flag that
  Param as changed whenever the Gather[Merge] node is rescanned.  This solution
  breaks an undocumented assumption made by the parallel executor logic, namely
  that all rescans of nodes below a Gather[Merge] will happen synchronously
  during the ReScan of the top node itself.  But that's fundamentally contrary
  to the design of the ExecReScan code, and so was doomed to fail someday anyway
  (even if you want to argue that the bug being fixed here wasn't a failure of
  that assumption).  A follow-on patch will address that issue.  In the
  meantime, the worst that's expected to happen is that given very bad timing
  luck, the leader might have to do all the work during a rescan, because
  workers think they have nothing to do, if they are able to start up before the
  eventual ReScan of the leader's parallel-aware table scan node has reset the
  shared scan state.  Although this problem exists in 9.6, there does not seem
  to be any way for it to manifest there.  Without GatherMerge, it seems that a
  plan tree that has a rescan-short-circuiting node below Gather will always
  also have one above it that will short-circuit in the same cases, preventing
  the Gather from being rescanned.  Hence we won't take the risk of
  back-patching this change into 9.6.  But v10 needs it.  Discussion:
  https://postgr.es/m/CAA4eK1JkByysFJNh9M349u_nNjqETuEnY_y1VUc_kJiU0bxtaQ@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/7df2c1f8daeb361133ac8bdeaf59ceb0484e315a

- Restore test case from a2b70c89ca1a5fcf6181d3c777d82e7b83d2de1b.  Revert the
  reversion commits a20aac890 and 9b644745c.  In the wake of commit 7df2c1f8d,
  we should get stable buildfarm results from this test; if not, I'd like to
  know sooner not later.  Discussion:
  https://postgr.es/m/CAA4eK1JkByysFJNh9M349u_nNjqETuEnY_y1VUc_kJiU0bxtaQ@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/6c2c5bea3cec4c874d1ee225bb6e222055c03d75

- Separate reinitialization of shared parallel-scan state from ExecReScan.
  Previously, the parallel executor logic did reinitialization of shared state
  within the ExecReScan code for parallel-aware scan nodes.  This is
  problematic, because it means that the ExecReScan call has to occur
  synchronously (ie, during the parent Gather node's ReScan call).  That is
  swimming very much against the tide so far as the ExecReScan machinery is
  concerned; the fact that it works at all today depends on a lot of fragile
  assumptions, such as that no plan node between Gather and a parallel-aware
  scan node is parameterized.  Another objection is that because ExecReScan
  might be called in workers as well as the leader, hacky extra tests are needed
  in some places to prevent unwanted shared-state resets.  Hence, let's separate
  this code into two functions, a ReInitializeDSM call and the ReScan call
  proper.  ReInitializeDSM is called only in the leader and is guaranteed to run
  before we start new workers.  ReScan is returned to its traditional function
  of resetting only local state, which means that ExecReScan's usual habits of
  delaying or eliminating child rescan calls are safe again.  As with the
  preceding commit 7df2c1f8d, it doesn't seem to be necessary to make these
  changes in 9.6, which is a good thing because the FDW and CustomScan APIs are
  impacted.  Discussion:
  https://postgr.es/m/CAA4eK1JkByysFJNh9M349u_nNjqETuEnY_y1VUc_kJiU0bxtaQ@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/41b0dd987d44089dc48e9c70024277e253b396b7

- Code review for nodeGatherMerge.c.  Comment the fields of GatherMergeState,
  and organize them a bit more sensibly.  Comment GMReaderTupleBuffer more
  usefully too.  Improve assorted other comments that were obsolete or just not
  very good English.  Get rid of the use of a GMReaderTupleBuffer for the leader
  process; that was confusing, since only the "done" field was used, and that in
  a way redundant with need_to_scan_locally.  In gather_merge_init, avoid
  calling load_tuple_array for already-known-exhausted workers.  I'm not sure if
  there's a live bug there, but the case is unlikely to be well tested due to
  timing considerations.  Remove some useless code, such as duplicating the
  tts_isempty test done by TupIsNull.  Remove useless initialization of ps.qual,
  replacing that with an assertion that we have no qual to check.  (If we did,
  the code would fail to check it.) Avoid applying heap_copytuple to a null
  tuple.  While that fails to crash, it's confusing and it makes the code less
  legible not more so IMO.  Propagate a couple of these changes into
  nodeGather.c, as well.  Back-patch to v10, partly because of the possibility
  that the gather_merge_init change is fixing a live bug, but mostly to keep the
  branches in sync to ease future bug fixes.
  https://git.postgresql.org/pg/commitdiff/04e9678614ec64ad9043174ac99a25b1dc45233a

- Improve code coverage of select_parallel test.  Make sure that rescans of
  parallel indexscans are tested.  Per code coverage report.
  https://git.postgresql.org/pg/commitdiff/4b1dd62a257a469f92fef4f4cce37beab6c0b98b

- Clean up shm_mq cleanup.  The logic around shm_mq_detach was a few bricks shy
  of a load, because (contrary to the comments for shm_mq_attach) all it did was
  update the shared shm_mq state.  That left us leaking a bit of process-local
  memory, but much worse, the on_dsm_detach callback for shm_mq_detach was still
  armed.  That means that whenever we ultimately detach from the DSM segment,
  we'd run shm_mq_detach again for already-detached, possibly long-dead queues.
  This accidentally fails to fail today, because we only ever re-use a shm_mq's
  memory for another shm_mq, and multiple detach attempts on the last such
  shm_mq are fairly harmless.  But it's gonna bite us someday, so let's clean it
  up.  To do that, change shm_mq_detach's API so it takes a shm_mq_handle not
  the underlying shm_mq.  This makes the callers simpler in most cases anyway.
  Also fix a few places in parallel.c that were just pfree'ing the handle
  structs rather than doing proper cleanup.  Back-patch to v10 because of the
  risk that the revenant shm_mq_detach callbacks would cause a live bug
  sometime.  Since this is an API change, it's too late to do it in 9.6.  (We
  could make a variant patch that preserves API, but I'm not excited enough to
  do that.) Discussion: https://postgr.es/m/8670.1504192177@sss.pgh.pa.us
  https://git.postgresql.org/pg/commitdiff/6708e447efb5046c95bdcf900b6da97f56f97ae8

- Avoid memory leaks when a GatherMerge node is rescanned.  Rescanning a
  GatherMerge led to leaking some memory in the executor's query-lifespan
  context, because most of the node's working data structures were simply
  abandoned and rebuilt from scratch.  In practice, this might never amount to
  much, given the cost of relaunching worker processes --- but it's still pretty
  messy, so let's fix it.  We can rearrange things so that the tuple arrays are
  simply cleared and reused, and we don't need to rebuild the TupleTableSlots
  either, just clear them.  One small complication is that because we might get
  a different number of workers on each iteration, we can't keep the old
  convention that the leader's gm_slots[] entry is the last one; the leader
  might clobber a TupleTableSlot that we need for a worker in a future
  iteration.  Hence, adjust the logic so that the leader has slot 0 always,
  while the active workers have slots 1..n.  Back-patch to v10 to keep all the
  existing versions of nodeGatherMerge.c in sync --- because of the renumbering
  of the slots, there would otherwise be a very large risk that any future
  backpatches in this module would introduce bugs.  Discussion:
  https://postgr.es/m/8670.1504192177@sss.pgh.pa.us
  https://git.postgresql.org/pg/commitdiff/2d44c58c79aeef2d376be0141057afbb9ec6b5bc

- Ensure SIZE_MAX can be used throughout our code.  Pre-C99 platforms may lack
  <stdint.h> and thereby SIZE_MAX.  We have a couple of places using the hack
  "(size_t) -1" as a fallback, but it wasn't universally available; which means
  the code added in commit 2e70d6b5e fails to compile everywhere.  Move that
  hack to c.h so that we can rely on having SIZE_MAX everywhere.  Per
  discussion, it'd be a good idea to make the macro's value safe for use in
  #if-tests, but that will take a bit more work.  This is just a quick expedient
  to get the buildfarm green again.  Back-patch to all supported branches, like
  the previous commit.  Discussion:
  https://postgr.es/m/15883.1504278595@sss.pgh.pa.us
  https://git.postgresql.org/pg/commitdiff/b79d69b087561eb6687373031a5098b0694f9ec6

- Make [U]INT64CONST safe for use in #if conditions.  Instead of using a cast to
  force the constant to be the right width, assume we can plaster on an L, UL,
  LL, or ULL suffix as appropriate.  The old approach to this is very hoary,
  dating from before we were willing to require compilers to have working int64
  types.  This fix makes the PG_INT64_MIN, PG_INT64_MAX, and PG_UINT64_MAX
  constants safe to use in preprocessor conditions, where a cast doesn't work.
  Other symbolic constants that might be defined using [U]INT64CONST are
  likewise safer than before.  Also fix the SIZE_MAX macro to be similarly safe,
  if we are forced to provide a definition for that.  The test added in commit
  2e70d6b5e happens to do what we want even with the hack "(size_t) -1"
  definition, but we could easily get burnt on other tests in future.
  Back-patch to all supported branches, like the previous commits.  Discussion:
  https://postgr.es/m/15883.1504278595@sss.pgh.pa.us
  https://git.postgresql.org/pg/commitdiff/9d6b160d7db76809f0c696d9073f6955dd5a973a

- Improve division of labor between execParallel.c and nodeGather[Merge].c.
  Move the responsibility for creating/destroying TupleQueueReaders into
  execParallel.c, to avoid duplicative coding in nodeGather.c and
  nodeGatherMerge.c.  Also, instead of having DestroyTupleQueueReader do
  shm_mq_detach, do it in the caller (which is now only ExecParallelFinish).
  This means execParallel.c does both the attaching and detaching of the
  tuple-queue-reader shm_mqs, which seems less weird than the previous
  arrangement.  These changes also eliminate a vestigial memory leak (of the
  pei->tqueue array).  It's now demonstrable that rescans of Gather or
  GatherMerge don't leak memory.  Discussion:
  https://postgr.es/m/8670.1504192177@sss.pgh.pa.us
  https://git.postgresql.org/pg/commitdiff/51daa7bdb39e1bdc31eb99fd3f54f61743ebb7ae

- Fix macro-redefinition warning on MSVC.  In commit 9d6b160d7, I tweaked
  pg_config.h.win32 to use "#define HAVE_LONG_LONG_INT_64 1" rather than
  defining it as empty, for consistency with what happens in an autoconf'd
  build.  But Solution.pm injects another definition of that macro into
  ecpg_config.h, leading to justifiable (though harmless) compiler whining.
  Make that one consistent too.  Back-patch, like the previous patch.
  Discussion:
  https://postgr.es/m/CAEepm=1dWsXROuSbRg8PbKLh0S=8Ou-V8sr05DxmJOF5chBxqQ@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/e451901804bd96a6b0fe3875b5c90aa0555c6a05

- Suppress compiler warnings in dshash.c.  Some compilers complain, not
  unreasonably, about left-shifting an int32 "1" and then assigning the result
  to an int64.  In practice I sure hope that this data structure never gets
  large enough that an overflow would actually occur; but let's cast the
  constant to the right type to avoid the hazard.  In passing, fix a typo in
  dshash.h.  Amit Kapila, adjusted as per comment from Thomas Munro.
  Discussion:
  https://postgr.es/m/CAA4eK1+5vfVMYtjK_NX8O3-42yM3o80qdqWnQzGquPrbq6mb+A@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/4faa1dc2eb02ba67303110e025d44abb40b12725

Robert Haas pushed:

- Push tuple limits through Gather and Gather Merge.  If we only need, say, 10
  tuples in total, then we certainly don't need more than 10 tuples from any
  single process.  Pushing down the limit lets workers exit early when possible.
  For Gather Merge, there is an additional benefit: a Sort immediately below the
  Gather Merge can be done as a bounded sort if there is an applicable limit.
  Robert Haas and Tom Lane Discussion:
  http://postgr.es/m/CA+TgmoYa3QKKrLj5rX7UvGqhH73G1Li4B-EKxrmASaca2tFu9Q@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/3452dc5240da43e833118484e1e9b4894d04431c

- Propagate sort instrumentation from workers back to leader.  Up until now,
  when parallel query was used, no details about the sort method or space used
  by the workers were available; details were shown only for any sorting done by
  the leader.  Fix that.  Commit 1177ab1dabf72bafee8f19d904cee3a299f25892 forced
  the test case added by commit 1f6d515a67ec98194c23a5db25660856c9aab944 to run
  without parallelism; now that we have this infrastructure, allow that again,
  with a little tweaking to make it pass with and without force_parallel_mode.
  Robert Haas and Tom Lane Discussion:
  http://postgr.es/m/CA+Tgmoa2VBZW6S8AAXfhpHczb=Rf6RqQ2br+zJvEgwJ0uoD_tQ@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/bf11e7ee2e3607bb67d25aec73aa53b2d7e9961b

- Expand partitioned tables in PartDesc order.  Previously, we expanded the
  inheritance hierarchy in the order in which find_all_inheritors had locked the
  tables, but that turns out to block quite a bit of useful optimization.  For
  example, a partition-wise join can't count on two tables with matching bounds
  to get expanded in the same order.  Where possible, this change results in
  expanding partitioned tables in *bound* order.  Bound order isn't well-defined
  for a list-partitioned table with a null-accepting partition or for a
  list-partitioned table where the bounds for a single partition are interleaved
  with other partitions.  However, when expansion in bound order is possible, it
  opens up further opportunities for optimization, such as strength-reducing
  MergeAppend to Append when the expansion order matches the desired sort order.
  Patch by me, with cosmetic revisions by Ashutosh Bapat.  Discussion:
  http://postgr.es/m/CA+TgmoZrKj7kEzcMSum3aXV4eyvvbh9WD=c6m=002WMheDyE3A@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/30833ba154e0c1106d61e3270242dc5999a3e4f3

- Introduce 64-bit hash functions with a 64-bit seed.  This will be useful for
  hash partitioning, which needs a way to seed the hash functions to avoid
  problems such as a hash index on a hash partitioned table clumping all values
  into a small portion of the bucket space; it's also useful for anything that
  wants a 64-bit hash value rather than a 32-bit hash value.  Just in case
  somebody wants a 64-bit hash value that is compatible with the existing 32-bit
  hash values, make the low 32-bits of the 64-bit hash value match the 32-bit
  hash value when the seed is 0.  Robert Haas and Amul Sul Discussion:
  http://postgr.es/m/CA+Tgmoafx2yoJuhCQQOL5CocEi-w_uG4S2xT0EtgiJnPGcHW3g@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/81c5e46c490e2426db243eada186995da5bb0ba7

- Try to repair poorly-considered code in previous commit.
  https://git.postgresql.org/pg/commitdiff/0d9506d125beef18247a5e38a219d3b23e2d312e

- Fix assorted carelessness about Datum vs. int64 vs. uint64.  Bugs introduced
  by commit 81c5e46c490e2426db243eada186995da5bb0ba7
  https://git.postgresql.org/pg/commitdiff/7b69b6ceb8047979ddf82af12ec1de143da62263

- Use group updates when setting transaction status in clog.  Commit
  0e141c0fbb211bdd23783afa731e3eef95c9ad7a introduced a mechanism to reduce
  contention on ProcArrayLock by having a single process clear XIDs in the
  procArray on behalf of multiple processes, reducing the need to hand the lock
  around.  A previous attempt to introduce a similar mechanism for
  CLogControlLock in ccce90b398673d55b0387b3de66639b1b30d451b crashed and
  burned, but the design problem which resulted in those failures is believed to
  have been corrected in this version.  Amit Kapila, with some cosmetic changes
  by me.  See the previous commit message for additional credits.  Discussion:
  http://postgr.es/m/CAA4eK1KudxzgWhuywY_X=yeSAhJMT4DwCjroV5Ay60xaeB2Eew@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/baaf272ac908ea27c09076e34f62c45fa7d1e448

- pg_dumpall: Add a -E flag to set the encoding, like pg_dump has.  Michael
  Paquier, reviewed by Fabien Coelho Discussion:
  http://postgr.es/m/CAB7nPqQcYWmrm2n-dVaMUhYPKFU_DxQwPuUGuC4ZF+8B=dS5xQ@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/84be67181aab22ea8723ba0625ee690223cd8785

- Tighten up some code in RelationBuildPartitionDesc.  This probably doesn't
  save anything meaningful in terms of performance, but making the code simpler
  is a good idea anyway.  Code by Beena Emerson, extracted from a larger patch
  by Jeevan Ladhe, slightly adjusted by me.  Discussion:
  http://postgr.es/m/CAOgcT0ONgwajdtkoq+AuYkdTPY9cLWWLjxt_k4SXue3eieAr+g@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/0cb8b7531db063bce7def2ef24f616285f1f4b04

Simon Riggs pushed:

- Add note about diskspace usage of pg_commit_ts.  Author: Thomas Munro
  https://git.postgresql.org/pg/commitdiff/abe85ef1d00187a42e7a757ea0413bc4965a4525

- Avoid race condition in logical replication test.  Wait for slot to become
  inactive before continuing.  Author: Petr Jelinek
  https://git.postgresql.org/pg/commitdiff/4f27c674fd9fb5ba1f2952e2db53886bb5954e8b

- Provisional list of Major Features.
  https://git.postgresql.org/pg/commitdiff/2f5ada2710d6e5a668d6d6b27f93ac545a01bafd

Álvaro Herrera pushed:

- Add a WAIT option to DROP_REPLICATION_SLOT.  Commit 9915de6c1cb2 changed the
  default behavior of DROP_REPLICATION_SLOT so that it would wait until any
  session holding the slot active would release it, instead of raising an error.
  But users are already depending on the original behavior, so revert to it by
  default and add a WAIT option to invoke the new behavior.  Per complaint from
  Simone Gotti, in Discussion:
  https://postgr.es/m/CAEvsy6Wgdf90O6pUvg2wSVXL2omH5OPC-38OD4Zzgk-FXavj3Q@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/be7161566db247fd519e1a888ea8cd36b3c72088

- Restore behavior for replication origin drop.  Do for replication origins what
  the previous commit did for replication slots: restore the original behavior
  of replication origin drop to raise an error rather than blocking, because
  users might be depending on the original behavior.  Maintain the blocking
  behavior when invoked internally from logical replication subscription
  handling.  Discussion:
  https://postgr.es/m/20170830133922.tlpo3lgfejm4n2cs@alvherre.pgsql
  https://git.postgresql.org/pg/commitdiff/a6979c3a68e2caa6696021f7a15278a99e7409a1

- Fix two-phase commit test for recovery mode.  The original code had a race
  condition because it never ensured the standby was caught up before
  proceeding; add a wait similar to every other place that does this.  Author:
  Michaël Paquier Discussion:
  https://postgr.es/m/CAB7nPqTm9p+LCm1mVJYvgpwagRK+uibT-pKq0O2-paOWxT62jw@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/89c59b742a7f89eb598a25b70aaa3ab97381f67d

== Pending Patches ==

Kyotaro HORIGUCHI sent in another revision of a patch to fix a race condition
between SELECT and ALTER TABLE NO INHERIT.

Kyotaro HORIGUCHI sent in another revision of a patch to retard restart LSN of a
slot when a segment starts with a contrecord and add some debugging code for
same.

Aaron Patters sent in a patch to custom allocators to libpq by adding
PQunescapeByteaConn for unescaping bytes given a connection and using the
configured malloc / free from the connection.

Kyotaro HORIGUCHI sent in a patch to show "aggressive" or not in vacuum
messages.

Craig Ringer sent in two revisions of a patch to show sockdir/hostname in
pg_regress startup output.

Michael Malis sent in two revisions of a patch to improve correlated partial
index cost estimates.

Masahiko Sawada sent in five more revisions of a patch to enable pgbench custom
initialization.

Rajkumar Raghuwanshi and Ashutosh Bapat traded patches to create an advanced
partition matching algorithm for partition-wise join.

Kyotaro HORIGUCHI sent in a patch to clean up the negative cache of pg_statistic
when dropping a relation and clean up the negative cache of pg_class when
dropping a schema.

Kyotaro HORIGUCHI sent in another revision of a patch to restrict maximum keep
segments by repslots.

Amit Langote, Robert Haas, and Ashutosh Bapat traded patches to expand single
inheritance child and expand inheritance by bound order.

Andrey Borodin sent in another revision of a patch to add hooks to watch for
changed pages.

Simon Riggs sent in a patch to add a "toast_tuple_target" table-level GUC which
specifies the threshold of tuple length at which PostgreSQL tries to move long
column values into TOAST tables.

Mithun Cy sent in another revision of a patch to cache data in GetSnapshotData()
for reuse.

Yangjie sent in another revision of a patch to implement hash partitioning.

Michaël Paquier sent in another revision of a patch to refactor error message
handling in pqAddTuple() and improve overflow checks of pqAddTuple() in libpq.

Dmitry Dolgov sent in another revision of a patch to implement generic type
subscripting.

Etsuro Fujita sent in another revision of a patch to add epqpath for foreign
joins.

Peter Geoghegan sent in another revision of a patch to add a bloom filter data
structure implementation and use same to build amcheck verification of indexes
against the heap.

Michaël Paquier sent in a patch to ensure that int64 is used in pg_rewind logs.

Dilip Kumar sent in another revision of a patch to improve bitmap costing for
lossy pages.

David Steele sent in two more revisions of a patch to update the low-level
backup documentation to match its actual behavior.

Nathan Bossart sent in five more revisions of a patch to enable users to specify
multiple tables in VACUUM commands.

David Steele sent in a patch to refactor OpenFile() permissions handling.

Etsuro Fujita sent in another revision of a patch to skip the
CheckValidResultRel checks for a target table if it's a foreign partition to
perform tuple-routing for.

Alexander Korotkov sent in another revision of a patch to change ALTER INDEX ...
SET STATISTICS ... to set or reset the behavior of a given expression in the
index.

Amit Kapila sent in another revision of his "group update clog" patch.

Peter Eisentraut sent in a patch to document and use SPI_result_code_string().

Peter Eisentraut sent in a patch to clean up places where global string
variables are initialized as empty strings ("") and then checked later with
strcmp(), instead of just using NULL.

Jing Wang sent in another revision of a patch to support to COMMENT ON DATABASE
CURRENT_DATABASE.

Haribabu Kommi sent in another revision of a patch to parallelize queries
containing initplans.

Amit Kapila sent in another revision of a patch to implement parallel append.

Jeff Davis sent in another revision of a patch to implement RANGE JOIN.

Beena Emerson sent in another revision of a patch to enable increasing the
default WAL segment size.

Peter Eisentraut sent in a patch to implement generated columns.

Kyotaro HORIGUCHI sent in a patch to allow multiple targets for VACUUM.

Andreas Karlsson sent in another revision of a patch to implement REINDEX
CONCURRENTLY.

Peter Geoghegan sent in a patch to remove replacement selection sort.

Peter Eisentraut sent in a patch to sync process names between ps and
pg_stat_activity.

Peter Eisentraut sent in another revision of a patch to support static
assertions in C++.

Peter Eisentraut sent in another revision of a patch to add background worker
type.

Andrey Borodin sent in a patch to add hooks to track changed pages for backup
purposes.

Tsutomu Yamada sent in another revision of a patch to add a CLUSTER command
progress monitor.

Jacob Champion sent in a patch to assert that the correct locks are held when
calling PageGetLSN().

Magnus Hagander sent in a patch to simplify the log_destination GUC by always
starting the logging collector.

Thomas Munro sent in another revision of a patch to implement parallel hash.

Andreas Karlsson sent in a patch to add GnuTLS support for SSL.

Magnus Hagander sent in another revision of a patch to add a function to move
the position of a replication slot.

Amit Langote sent in another revision of a patch to teach pg_inherits.c a bit
about partitioning, allow locking only partitioned children in partition tree,
defer opening and locking partitions to set_append_rel_size, change the
interfaces for partition_bound_{cmp/bsearch}, addition to partition.c interface
for partition-pruning, and make some planner-side changes for partition-pruning.

Haribabu Kommi sent in another revision of a patch to implement pluggable
storage.

Michaël Paquier sent in another revision of a patch to enable creating backup
history files for backups taken from standbys.

Thomas Munro sent in another revision of a patch to allow SERIALIZABLE isolation
level with parallel query.

Haribabu Kommi sent in another revision of a patch to make parallel eligible for
utility commands underneath queries.

Konstantin Knizhnik sent in another revision of a patch to implement surjective
functional indexes.

Etsuro Fujita sent in another revision of a patch to update comment in
ExecPartitionCheck.

Thomas Munro sent in another revision of a patch to remove obsolete SLRU
wrapping and warnings from predicate.c.

Tanay Varma sent in a patch to enable Visual Studio 2017 Build Support.

Antonin Houska sent in another revision of a patch to add some tools to help
with partition-wise join for join between (declaratively) partitioned tables.

Antonin Houska sent in another revision of a patch to implement partition-wise
join for join between (declaratively) partitioned tables.

Jeff Janes sent in a patch to ensure that pg_basebackup throttling throttles as
promised.

Andres Freund sent in another revision of a patch to make JIT compiling
expressions/deform + inlining available.

Simon Riggs and Tom Lane traded patches to fix a savepoint bug.

Michaël Paquier sent in another revision of a patch to change detection of
corrupted 2PC files to be FATAL and minimize the window between history file and
end-of-recovery.

Konstantin Knizhnik sent in another revision of a patch to optimize secondary
indexes.

Emre Hasegeli sent in another revision of a patch to improve geometric types.

David Steele sent in a patch to exclude pg_internal.init from base backup.

Alik Khilazhev sent in another revision of a patch to add a Zipfian distribution
in pgbench.

Nikolay Shaplov sent in another revision of a patch to move all am-related
reloption code into src/backend/access/[am-name] and get rid of relopt_kind for
custom AM.

Thomas Munro sent in another revision of a patch to add an optional message to
user when terminating/cancelling backend.

Amit Kapila and Thomas Munro traded patches to fix warnings and typo in dshash.