== PostgreSQL Weekly News - February 28 2016 ==

== PostgreSQL Weekly News - February 28 2016 ==

== PostgreSQL Product News ==

Ora2PG 17.0 a migration system for moving from Oracle and MySQL to
PostgreSQL, released.
http://ora2pg.darold.net/

pgBadger 8.0, a parallel PostgreSQL log analyzer written in Perl, released:
https://github.com/dalibo/pgbadger/blob/master/ChangeLog
Development:
https://github.com/dalibo/pgbadger/

pgbouncer 1.7.2, a light-weight connection pooler for PostgreSQL,
released.
https://pgbouncer.github.io/changelog.html

== PostgreSQL Jobs for February ==

http://archives.postgresql.org/pgsql-jobs/2016-02/

== PostgreSQL Local ==

The first pan-Asian PostgreSQL conference will be held March 17-19,
2016 in Singapore.  Registration is open.
http://2016.pgday.asia/

Nordic PGDay 2016 is a one day one track conference which will be held in
Helsinki, Finland, on March 17, 2016.  Registration is still open.
http://2016.nordicpgday.org/

PGDay Paris 2016, on March 31 in Paris, has opened registration.
http://www.pgday.paris/registration/ (scroll down for English)

The 8th PostgreSQL Session will be held on April 6th, 2016, in Lyon,
France.  The CfP is open until February 29, 2016 at call-for-paper
<AT> postgresql-sessions <DOT> org.

PGConf US 2016 will take place April 18-20, 2016 in NYC.  Registration
is open.
http://www.pgconf.us/2016/

LinuxFest Northwest will take place April 23-24, 2016 at Bellingham
Technical College in Bellingham, Washington, USA.  The CfP is now
open.
http://www.linuxfestnorthwest.org/2016/present

FOSS4G NA, will be held May 2-5, 2016 in Raleigh, North Carolina.
The CfP is still open.
https://2016.foss4g-na.org/cfp

PGCon 2016 will be held May 17-21, 2016 in Ottawa.
http://www.pgcon.org/

This year's Swiss PGDay will be held on June 24, 2016 at the
University of Applied Sciences in Rapperswil (Switzerland).
The CfP is open.
http://www.pgday.ch/

"5432 ... Meet us!", will take place in Milan, Italy on June 28-29, 2016.
The CfP is open until February 28th, 2016.
http://5432meet.us/

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david@fetter.org, German language
to pwn@pgug.de, Italian language to pwn@itpug.org.  Spanish language
to pwn@arpug.com.ar.

== Applied Patches ==

Andres Freund pushed:

- Fix wrong keysize in PrivateRefCountHash creation.  In 4b4b680c3 I
  accidentally used sizeof(PrivateRefCountArray) instead of
  sizeof(PrivateRefCountEntry) when creating the refcount overflow
  hashtable. As the former is bigger than the latter, this luckily
  only resulted in a slightly increased memory usage when many buffers
  are pinned in a backend.  Reported-By: Takashi Horikawa Discussion:
  73FA3881462C614096F815F75628AFCD035A48C3@BPXM01GP.gisp.nec.co.jp
  Backpatch: 9.5, where thew new ref count infrastructure was
  introduced
  http://git.postgresql.org/pg/commitdiff/ea56b06cf77a6932a74f9d4ec6c950a333d1527d

Tom Lane pushed:

- Remove redundant PGPROC.lockGroupLeaderIdentifier field.  We don't
  really need this field, because it's either zero or redundant with
  PGPROC.pid.  The use of zero to mark "not a group leader" is not
  necessary since we can just as well test whether lockGroupLeader is
  NULL.  This does not save very much, either as to code or data, but
  the simplification seems worthwhile anyway.
  http://git.postgresql.org/pg/commitdiff/73bf8715aa7430bd003516bde448507fbe789c05

- Create a function to reliably identify which sessions block which
  others.  This patch introduces "pg_blocking_pids(int) returns
  int[]", which returns the PIDs of any sessions that are blocking the
  session with the given PID.  Historically people have obtained such
  information using a self-join on the pg_locks view, but it's
  unreasonably tedious to do it that way with any modicum of
  correctness, and the addition of parallel queries has pretty much
  broken that approach altogether.  (Given some more columns in the
  view than there are today, you could imagine handling parallel-query
  cases with a 4-way join; but ugh.) The new function has the
  following behaviors that are painful or impossible to get right via
  pg_locks: 1. Correctly understands which lock modes block which
  other ones.  2. In soft-block situations (two processes both waiting
  for conflicting lock modes), only the one that's in front in the
  wait queue is reported to block the other.  3. In parallel-query
  cases, reports all sessions blocking any member of the given PID's
  lock group, and reports a session by naming its leader process's
  PID, which will be the pg_backend_pid() value visible to clients.
  The motivation for doing this right now is mostly to fix the
  isolation tests.  Commit 38f8bdcac4982215beb9f65a19debecaf22fd470
  lobotomized isolationtester's is-it-waiting query by removing its
  ability to recognize nonconflicting lock modes, as a crude
  workaround for the inability to handle soft-block situations
  properly.  But even without the lock mode tests, the old query was
  excessively slow, particularly in CLOBBER_CACHE_ALWAYS builds; some
  of our buildfarm animals fail the new deadlock-hard test because the
  deadlock timeout elapses before they can probe the waiting status of
  all eight sessions.  Replacing the pg_locks self-join with use of
  pg_blocking_pids() is not only much more correct, but a lot faster:
  I measure it at about 9X faster in a typical dev build with Asserts,
  and 3X faster in CLOBBER_CACHE_ALWAYS builds.  That should provide
  enough headroom for the slower CLOBBER_CACHE_ALWAYS animals to pass
  the test, without having to lengthen deadlock_timeout yet more and
  thus slow down the test for everyone else.
  http://git.postgresql.org/pg/commitdiff/52f5d578d6c29bf254e93c69043b817d4047ca67

Noah Misch pushed:

- MSVC: Clean tmp_check directory of pg_controldata test suite.
  Back-patch to 9.4, where the suite was introduced.
  http://git.postgresql.org/pg/commitdiff/41635887835d54715a2fd4745dbf0fdb99f57f4f

- Clean the last few TAP suite tmp_check directories.  Back-patch to
  9.5, where the suites were introduced.
  http://git.postgresql.org/pg/commitdiff/25924ac47abde5330ca502e23796e9d37cd1ae68

Robert Haas pushed:

- Enable parallelism for prepared statements and extended query
  protocol.  Parallel query can't handle running a query only
  partially rather than to completion.  However, there seems to be no
  way to run a statement prepared via SQL PREPARE other than to
  completion, so we can enable it there without a problem.  The
  situation is more complicated for the extend query protocol.  libpq
  seems to provide no way to send an Execute message with a non-zero
  rowcount, but some other client might.  If that happens, and a
  parallel plan was chosen, we'll execute the parallel plan without
  using any workers, which may be somewhat inefficient but should
  still work.  Hopefully this won't be a problem; users can always set
  max_parallel_degree=0 to avoid choosing parallel plans in the first
  place.  Amit Kapila, reviewed by me.
  http://git.postgresql.org/pg/commitdiff/57a6a72b6bc98f3003e87bc31de4b9c2c89fe019

- Add new FDW API to test for parallel-safety.  This is basically a
  bug fix; the old code assumes that a ForeignScan is always
  parallel-safe, but for postgres_fdw, for example, this is definitely
  false.  It should be true for file_fdw, though, since a worker can
  read a file from the filesystem just as well as any other backend
  process.  Original patch by Thomas Munro.  Documentation, and
  changes to the comments, by me.
  http://git.postgresql.org/pg/commitdiff/35746bc348b6bf1f690fe17f4f80cfb68e22f504

- On second thought, disable parallelism for prepared statements.
  CREATE TABLE .. AS EXECUTE can turn an apparently read-only query
  into a write operation, which parallel query can't handle.  It's a
  bit of a shame that requires us to avoid parallel query for queries
  prepared via PREPARE in all cases, but for right now it does.
  http://git.postgresql.org/pg/commitdiff/7bea19d0a9d3e6975418ffe685fb510bd31ab434

- Respect TEMP_CONFIG when running contrib regression tests.  Thomas
  Munro
  http://git.postgresql.org/pg/commitdiff/9117985b6ba9beda4f280f596035649fc23b6233

Álvaro Herrera pushed:

- Fix typos Backpatch to: 9.4
  http://git.postgresql.org/pg/commitdiff/343f709c0691abdc779bec91fa3587dda259808e

- Add README in src/test and src/test/modules.  Author: Craig Ringer
  Reviewed by: Michaël Paquier
  http://git.postgresql.org/pg/commitdiff/bda0b081984011ba5347bf3eecc95b71833de082

- Add POD docs to PostgresNode.  Also, the dump_info method got split
  into another method that returns the stuff as a string instead of
  just printing it to stdout.  Add a new README in src/test/perl too.
  Author: Craig Ringer Reviewed by: Michaël Paquier
  http://git.postgresql.org/pg/commitdiff/e64009303d4e2434abafbdafe4d571cc4f279d39

- Move some code from RewindTest into PostgresNode.  Some code in the
  RewindTest test suite is more generally useful than just for that
  suite, so put it where other test suites can reach it.  Some
  postgresql.conf parameters change their default values when a
  cluster is initialized with 'allows_streaming' than the previous
  behavior; most notably, autovacuum is no longer turned off.  (Also,
  we no longer call pg_ctl promote with -w, but that flag doesn't
  actually do anything in promote so there's no behavior change.)
  Author: Michael Paquier
  http://git.postgresql.org/pg/commitdiff/89ac7004dadf4116d9b180bb5ff64b64bfce94b1

- Apply last revision of recovery patch.  I applied the
  previous-to-last revision of Michaël's submitted patch instead of
  the last; these two tweaks pointed out by Craig were left out of the
  previous commit by accident.
  http://git.postgresql.org/pg/commitdiff/74d58425c70f9f7f1cb5d7f428ceaa1b56aa081d

- Add a test framework for recovery.  This long-awaited framework is
  an expansion of the existing PostgresNode stuff to support
  additional features for recovery testing; the recovery tests
  included in this commit are a starting point that cover some of the
  recovery features we have.  More scripts are expected to be added
  later.  Author: Michaël Paquier, a bit of help from Amir Rohan.
  Reviewed by: Amir Rohan, Stas Kelvich, Kyotaro Horiguchi, Victor
  Wagner, Craig Ringer, Álvaro Herrera.  Discussion:
  http://www.postgresql.org/message-id/CAB7nPqTf7V6rswrFa=q_rrWeETUWagP=h8LX8XAov2Jcxw0DRg@mail.gmail.com
  Discussion:
  http://www.postgresql.org/message-id/trinity-b4a8035d-59af-4c42-a37e-258f0f28e44a-1443795007012@3capp-mailcom-lxa08
  http://git.postgresql.org/pg/commitdiff/49148645f7f30f461b09618b5342b37425f33b22

- Add isolationtester spec for old heapam.c bug.  In 0e5680f4737a, I
  fixed a bug in heapam that caused spurious deadlocks when multiple
  updates concurrently attempted to modify the old version of an
  updated tuple whose new version was key-share locked.  I proposed an
  isolationtester spec file that reproduced the bug, but back then
  isolationtester wasn't mature enough to be able to run it.  Now that
  38f8bdcac498 is in the tree, we can have this spec file too.
  Discussion:
  https://www.postgresql.org/message-id/20141212205254.GC1768%40alvh.no-ip.org
  http://git.postgresql.org/pg/commitdiff/c9578135f769072e2597b88402f256a398279c91

Andrew Dunstan pushed:

- Respect TEMP_CONFIG when pg_regress_check and friends are called.
  This reverts commit 9117985b6ba9beda4f280f596035649fc23b6233 in
  favor of a more general solution.
  http://git.postgresql.org/pg/commitdiff/87cc6b57a911bf439ff9e53f1eddba58e9749ffe

- Allow multiple --temp-config arguments to pg_regress.  This means that
  if, for example, TEMP_CONFIG is set and a Makefile explicitly sets a
  temp-config file, both will now be used.  Patch from John Gorman.
  http://git.postgresql.org/pg/commitdiff/26fdff1b8f76f3c47d4e19be7c4aef3cdcd3393c

== Rejected Patches (for now) ==

No one was disappointed this week :-)

== Pending Patches ==

Amit Kapila sent in another revision of a patch to extend
pg_stat_activity with wait_type_event.

Corey Huinker sent in another revision of a patch to add \gexec to
psql.

Rushabh Lathia sent in another revision of a patch to help fix some
slowness in FDW DML.

Michaël Paquier sent in another revision of a patch to add new
authentication methods with SCRAM as one example.

Vitaly Burovoy sent in a patch to fix handling of negative years.

Ashutosh Bapat sent in another revision of a patch to push sorted
joins down to FDWs.

Fujii Masao sent in a patch to add tab completion in psql to CREATE
USER MAPPING.

Michaël Paquier sent in a pair of patches, one which extends
XLogInsert() with an extra argument for flags, the other which
introduces XLogInsertExtended with this extra argument and leaves
XLogInsert() alone.

Michaël Paquier sent in another revision of a patch to fix a potential
data loss bug on ext4 filesystems.

Craig Ringer sent in another revision of a patch to implement failover
slots.

Tomas Vondra and Mark Dilger traded patches to improve GROUP BY
estimation.

Tomas Vondra and Kyotaro HORIGUCHI traded patches to allow index-only
scans with partial indexes.

Jim Nasby sent in another revision of a patch to convert PL/Tcl from
strings to objects.

Thomas Munro sent in two more revisions of a patch to make the
PostgreSQL in parallel mode safer on PPC.

Teodor Sigaev sent in two more revisions of a patch to fix a GIN
corruption bug.

Vitaly Burovoy sent in a patch to fix some overflows in timstamp[tz].

Iacob Catalin and Pavel Stěhule traded patches to add an ereport
function to PL/PythonU.

Mithun Cy sent in another revision of a patch to cache data in
GetSnapshotData().

Kyotaro HORIGUCHI and SAWADA Masahiko traded patches to support N>1
synchronous standby servers.

Petr Jelínek sent in another revision of a patch to add generic WAL
messages.

Julien Rouhaud sent in a patch to ensure that the CREATE OPERATOR
CLASS documentation mentions that BRIN indexes also support the
STORAGE parameter.

Kyotaro HORIGUCHI sent in a patch to fix wrong comments for PQmblen()
and PQdsplen().

Kyotaro HORIGUCHI sent in a patch to fix identifier completion with
multibyte characters.

Peter Eisentraut sent in a patch to add table qualifications to some
tags in pg_dump.

Kyotaro HORIGUCHI sent in another revision of a patch to add "IF [NOT]
EXISTS" support to psql's tab completion.

Michaël Paquier sent in four more revisions of a patch to add new
regression tests for recovery, etc.

Peter Eisentraut sent in a patch to introduce new configuration
parameters syslog_sequence_numbers and syslog_split_lines.

Amit Kapila sent in another revision of a patch to speed up CLOG
access.

Vitaly Burovoy sent in a patch to allow infinite values in
to_timestamp.

Pavel Stěhule sent in another revision of a patch to add a raw format
to COPY.

Vinayak Pokure sent in another revision of a patch to add a vacuum
progress checker.

Roma Sokolov sent in two revisions of a patch to fix DROP OPERATOR to
reset links to itself on commutator and negator.

Ivan Kartyshov sent in two revisions of a patch to add a
pg_oldest_xlog_location() function.

Simon Riggs sent in two revisions of a patch to fix an issue with
relcache invalidation on a physical replica.

Amit Langote sent in a patch to fix a typo in
src/backend/utils/init/postinit.c.

Joe Conway sent in two more revisions of a patch to add control data
functions.

Tom Lane sent in a WIP patch to path-ify the upper planner.  And there
was much rejoicing.

Jim Nasby sent in a patch to improve error handling in PL/Tcl.