== PostgreSQL Weekly News - February 15 2015 ==

== PostgreSQL Weekly News - February 15 2015 ==

== PostgreSQL Product News ==

pgpool-II versions 3.4.1, 3.3.5, 3.2.10, 3.1.13, and 3.0.17 released
with pgpoolAdmin 3.4.1.
pgpool-II is a connection pooler and replication system for PostgreSQL.
http://pgpool.net/mediawiki/index.php/Downloads
http://pgpool.net/mediawiki/index.php/Yum_Repository

psycopg2 2.6 and 2.5.5 Python connector for PostgreSQL, released.
http://initd.org/psycopg/articles/2015/02/09/psycopg-26-and-255-released/

== PostgreSQL Jobs for February ==

http://archives.postgresql.org/pgsql-jobs/2015-02/threads.php

== PostgreSQL Local ==

The Melbourne PostgreSQL meetup on February 18, 2015 will be hosting
Gabriele Bartolini on PostgreSQL 9.4 for devops.  Details below, and
R, SVP.
http://www.meetup.com/melpug/events/219082475/

pgDaySF 2015 will be held March 10, 2015 in Burlingame, California.
http://sfpostgres.org/pgday-sf-2015-call-for-speakers-and-sponsors/

Registration for Nordic PostgreSQL Day, March 11, 2015 in  Copenhagen,
Denmark, is open.
http://2015.nordicpgday.org/registration/

PGConf US 2015 takes place March 25-27, 2015 in NYC.
http://nyc.pgconf.us/2015/

PGCon 2015 is June 16-20 in Ottawa, Canada.
http://www.pgcon.org/2015/

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david@fetter.org, German language
to pwn@pgug.de, Italian language to pwn@itpug.org.  Spanish language
to pwn@arpug.com.ar.

== Applied Patches ==

Fujii Masao pushed:

- Move pg_lzcompress.c to src/common.  The meta data of PGLZ
  symbolized by PGLZ_Header is removed, to make the compression and
  decompression code independent on the backend-only varlena facility.
  PGLZ_Header is being used to store some meta data related to the
  data being compressed like the raw length of the uncompressed record
  or some varlena-related data, making it unpluggable once PGLZ is
  stored in src/common as it contains some backend-only code paths
  with the management of varlena structures. The APIs of PGLZ are
  reworked at the same time to do only compression and decompression
  of buffers without the meta-data layer, simplifying its use for a
  more general usage.  On-disk format is preserved as well, so there
  is no incompatibility with previous major versions of PostgreSQL for
  TOAST entries.  Exposing compression and decompression APIs of pglz
  makes possible its use by extensions and contrib modules. Especially
  this commit is required for upcoming WAL compression feature so that
  the WAL reader facility can decompress the WAL data by using
  pglz_decompress.  Michael Paquier, reviewed by me.
  http://git.postgresql.org/pg/commitdiff/40bede5477bb5bce98ce9548841cb414634c26f7

Heikki Linnakangas pushed:

- Fix MSVC build.  When I moved pg_crc.c from src/port to src/common,
  I forgot to modify MSVC build script accordingly.
  http://git.postgresql.org/pg/commitdiff/cc761b170c5e7b4ef22ed918f4785ec1fabe62cd

- Move pg_crc.c to src/common, and remove pg_crc_tables.h.  To get CRC
  functionality in a client program, you now need to link with
  libpgcommon instead of libpgport. The CRC code has nothing to do
  with portability, so libpgcommon is a better home. (libpgcommon
  didn't exist when pg_crc.c was originally moved to src/port.) Remove
  the possibility to get CRC functionality by just #including
  pg_crc_tables.h. I'm not aware of any extensions that actually did
  that and couldn't simply link with libpgcommon.  This also moves the
  pg_crc.h header file from src/include/utils to src/include/common,
  which will require changes to any external programs that currently
  does #include "utils/pg_crc.h". That seems acceptable, as
  include/common is clearly the right home for it now, and the change
  needed to any such programs is trivial.
  http://git.postgresql.org/pg/commitdiff/c619c2351f7ec429b6ddce519c939f7b8465d711

- Speed up CRC calculation using slicing-by-8 algorithm.  This speeds
  up WAL generation and replay. The new algorithm is significantly
  faster with large inputs, like full-page images or when inserting
  wide rows. It is slower with tiny inputs, i.e. less than 10 bytes or
  so, but the speedup with longer inputs more than make up for that.
  Even small WAL records at least have 24 byte header in the front.
  The output is identical to the current byte-at-a-time computation,
  so this does not affect compatibility. The new algorithm is only
  used for the CRC-32C variant, not the legacy version used in tsquery
  or the "traditional" CRC-32 used in hstore and ltree. Those are not
  as performance critical, and are usually only applied over small
  inputs, so it seems better to not carry around the extra lookup
  tables to speed up those rare cases.  Abhijit Menon-Sen
  http://git.postgresql.org/pg/commitdiff/025c02420de990c15a90e9e3f86fcfbc5b59ee88

- Simplify waiting logic in reading from / writing to client.  The
  client socket is always in non-blocking mode, and if we actually
  want blocking behaviour, we emulate it by sleeping and retrying. But
  we have retry loops at different layers for reads and writes, which
  was confusing.  To simplify, remove all the sleeping and retrying
  code from the lower levels, from be_tls_read and secure_raw_read and
  secure_raw_write, and put all the logic in secure_read() and
  secure_write().
  http://git.postgresql.org/pg/commitdiff/80788a431e9bff06314a054109fdea66ac538199

- Simplify the way OpenSSL renegotiation is initiated in server.  At
  least in all modern versions of OpenSSL, it is enough to call
  SSL_renegotiate() once, and then forget about it. Subsequent
  SSL_write() and SSL_read() calls will finish the handshake.  The
  SSL_set_session_id_context() call is unnecessary too. We only have
  one SSL context, and the SSL session was created with that to begin
  with.
  http://git.postgresql.org/pg/commitdiff/272923a0a6956187471df4f032eee06559520390

- Fix broken #ifdef for __sparcv8.  Rob Rowan. Backpatch to all
  supported versions, like the patch that added the broken #ifdef.
  http://git.postgresql.org/pg/commitdiff/33e879c4e9f485405fc19b663392d8635714d020

Tom Lane pushed:

- Minor cleanup/code review for "indirect toast" stuff.  Fix some
  issues I noticed while fooling with an extension to allow an
  additional kind of toast pointer.  Much of this is just comment
  improvement, but there are a couple of actual bugs, which might or
  might not be reachable today depending on what can happen during
  logical decoding.  An example is that toast_flatten_tuple() failed
  to cover the possibility of an indirection pointer in its input.
  Back-patch to 9.4 just in case that is reachable now.  In HEAD, also
  correct some really minor issues with recent compression
  reorganization, such as dangerously underparenthesized macros.
  http://git.postgresql.org/pg/commitdiff/bc4de01db3a210e988dc88b585d7c38e6e7054c7

- Fix GEQO to not assume its join order heuristic always works.  Back
  in commit 400e2c934457bef4bc3cc9a3e49b6289bd761bc0 I rewrote GEQO's
  gimme_tree function to improve its heuristic for modifying the given
  tour into a legal join order.  In what can only be called a fit of
  hubris, I supposed that this new heuristic would *always* find a
  legal join order, and ripped out the old logic that allowed
  gimme_tree to sometimes fail.  The folly of this is exposed by bug
  #12760, in which the "greedy" clumping behavior of merge_clump() can
  lead it into a dead end which could only be recovered from by
  un-clumping.  We have no code for that and wouldn't know exactly
  what to do with it if we did.  Rather than try to improve the
  heuristic rules still further, let's just recognize that it *is* a
  heuristic and probably must always have failure cases.  So, put back
  the code removed in the previous commit to allow for failure (but
  comment it a bit better this time).  It's possible that this code
  was actually fully correct at the time and has only been broken by
  the introduction of LATERAL.  But having seen this example I no
  longer have much faith in that proposition, so back-patch to all
  supported branches.
  http://git.postgresql.org/pg/commitdiff/1a179f36f7c612d3560a72396c559cd876dfab6b

- Fix pg_dump's heuristic for deciding which casts to dump.  Back in
  2003 we had a discussion about how to decide which casts to dump.
  At the time pg_dump really only considered an object's containing
  schema to decide what to dump (ie, dump whatever's not in
  pg_catalog), and so we chose a complicated idea involving whether
  the underlying types were to be dumped (cf commit
  a6790ce85752b67ad994f55fdf1a450262ccc32e).  But users are allowed to
  create casts between built-in types, and we failed to dump such
  casts.  Let's get rid of that heuristic, which has accreted even
  more ugliness since then, in favor of just looking at the cast's OID
  to decide if it's a built-in cast or not.  In passing, also fix some
  really ancient code that supposed that it had to manufacture a
  dependency for the cast on its cast function; that's only true when
  dumping from a pre-7.3 server.  This just resulted in some wasted
  cycles and duplicate dependency-list entries with newer servers, but
  we might as well improve it.  Per gripes from a number of people,
  most recently Greg Sabino Mullane.  Back-patch to all supported
  branches.
  http://git.postgresql.org/pg/commitdiff/9feefedf9e92066fa6609d1e1e17b4892d81716f

- Fix more memory leaks in failure path in buildACLCommands.  We
  already had one go at this issue in commit d73b7f973db5ec7e, but we
  failed to notice that buildACLCommands also leaked several
  PQExpBuffers along with a simply malloc'd string.  This time let's
  try to make the fix a bit more future-proof by eliminating the
  separate exit path.  It's still not exactly critical because pg_dump
  will curl up and die on failure; but since the amount of the
  potential leak is now several KB, it seems worth back-patching as
  far as 9.2 where the previous fix landed.  Per Coverity, which
  evidently is smarter than clang's static analyzer.
  http://git.postgresql.org/pg/commitdiff/9179444d070713f53a97bc8f5c898a8304424c4d

- Fix minor memory leak in ident_inet().  We'd leak the ident_serv
  data structure if the second pg_getaddrinfo_all (the one for the
  local address) failed.  This is not of great consequence because a
  failure return here just leads directly to backend exit(), but if
  this function is going to try to clean up after itself at all, it
  should not have such holes in the logic.  Try to fix it in a
  future-proof way by having all the failure exits go through the same
  cleanup path, rather than "optimizing" some of them.  Per Coverity.
  Back-patch to 9.2, which is as far back as this patch applies
  cleanly.
  http://git.postgresql.org/pg/commitdiff/58146d35deed3bd3e05ce4b5ba529e4e4a369ddb

- Fix missing PQclear() in libpqrcv_endstreaming().  This omission
  leaked one PGresult per WAL streaming cycle, which possibly would
  never be enough to notice in the real world, but it's still a leak.
  Per Coverity.  Back-patch to 9.3 where the error was introduced.
  http://git.postgresql.org/pg/commitdiff/4f38a281a3d4a7cbcf9de2f676d44336d57b7717

- Avoid returning undefined bytes in chkpass_in().  We can't really
  fix the problem that the result is defined to depend on random(), so
  it is still going to fail the "unstable input conversion" test in
  parse_type.c.  However, we can at least satify valgrind.  (It looks
  like this code used to be valgrind-clean, actually, until somebody
  did a careless s/strncpy/strlcpy/g on it.) In passing, let's just
  make real sure that chkpass_out doesn't overrun its output buffer.
  No need for backpatch, I think, since this is just to satisfy
  debugging tools.  Asif Naeem
  http://git.postgresql.org/pg/commitdiff/80986e85aafd64165efc7f45b330f2f213713032

Michael Meskes pushed:

- Fixed array handling in ecpg.  When ecpg was rewritten to the new
  protocol version not all variable types were corrected. This patch
  rewrites the code for these types to fix that. It also fixes the
  documentation to correctly tell the status of array handling.
  http://git.postgresql.org/pg/commitdiff/1f393fc923ec77f25fd37e16fd8ccb480df82ebb

Andres Freund pushed:

- Fix typo in logicaldecoding.sgml.  Author: Tatsuo Ishii.  Backpatch
  to 9.4, where logical decoding was introduced.
  http://git.postgresql.org/pg/commitdiff/8785e6e378f919827d851926ea0f400f08afd477

Bruce Momjian pushed:

- pg_upgrade:  preserve freeze info for postgres/template1 dbs.
  pg_database.datfrozenxid and pg_database.datminmxid were not
  preserved for the 'postgres' and 'template1' databases.  This could
  cause missing clog file errors on access to user tables and indexes
  after upgrades in these databases.  Backpatch through 9.0
  http://git.postgresql.org/pg/commitdiff/866f3017a88b9cb639f05c9ad413e5eafe0266e3

- pg_upgrade:  improve checksum mismatch error message.  Patch by Greg
  Sabino Mullane, slight adjustments by me
  http://git.postgresql.org/pg/commitdiff/dc01efa5ccdd9804b0c97d4d9b651ba9360ebb11

- pg_upgrade:  quote directory names in delete_old_cluster script.
  This allows the delete script to properly function when special
  characters appear in directory paths, e.g. spaces.  Backpatch
  through 9.0
  http://git.postgresql.org/pg/commitdiff/056764b10248bff702d9d7b8b97690668eaf1c93

Peter Eisentraut pushed:

- pg_regress: Write processed input/*.source into output dir.  Before,
  it was writing the processed files into the input directory, which
  is incorrect in a vpath build.
  http://git.postgresql.org/pg/commitdiff/64cdbbc48cade73d7b0831444a62e19fd4a342f8

Andrew Dunstan pushed:

- Minor housekeeping on JSON docs.
  http://git.postgresql.org/pg/commitdiff/c5b31e29f0128b1c3c1caeeef81b0986cebdcd03

== Rejected Patches (for now) ==

No one was disappointed this week :-)

== Pending Patches ==

Tatsuo Ishii and Michael Paquier traded patches to fix some
incompatible options in pgbench.

David Steele sent in two more revisions of a patch to implement
pgaudit.

Jan Urbański sent in two revisions of a patch to fix broken
multi-threaded SSL callback handling in libpq.

Rahila Syed sent in three more revisions of a patch to implement
compression of full-page writes.

Peter Geoghegan and Heikki Linnakangas traded patches to implement
INSERT ... ON CONFLICT {UPDATE | IGNORE}.

Heikki Linnakangas sent in two revisions of a patch to use use Intel
SSE4.2 CRC instructions where available.

Michael Paquier sent in three more revisions of a patch to implement
table-level autovacuum_min_duration.

Michael Paquier sent in a patch to add a wal_retrieve_retry_interval.

Kyotaro HORIGUCHI sent in two revisions of a patch to fix an issue
where pg_basebackup fails silently.

Tom Lane sent in two revisions of a patch to manipulate complex types
as non-contiguous structures in-memory.

Etsuro Fujita sent in another revision of a patch to fix
ExplainModifyTarget to work as expected.

Michael Paquier sent in a patch to add palloc_extended for frontend
and backend, and another to rework handling of OOM when allocating
record buffer in XLOG reader.

Anastasia Lubennikova sent in two more revisions of a patch to allow
index-only scans for GiST indexes.

Emre Hasegeli sent in another revision of a patch to add a BRIN range
operator class.

Robert Haas sent in two more revisions of a patch to implement
parallel mode and parallel contexts.

Marco Nenciarini sent in another revision of a patch to implement
file-based incremental backup.

Marco Nenciarini sent in another revision of a patch to fix
pg_check_dir.

Michael Paquier sent in a patch to fix an odd assertion using
VACOPT_FREEZE in vacuum.c.

Teodor Sigaev sent in a patch to implement some of the intarray
features for all arrays.

Teodor Sigaev sent in another revision of a patch to add a compress
method for SP-GiST.

Heikki Linnakangas sent in another revision of a patch to redesign
checkpoint_segments.

Heikki Linnakangas sent in a patch to refactor unit conversions.

Atri Sharma sent in another revision of a patch to implement UPDATE
... SET(*) =.

Asif Naeem sent in a patch to fix chkpass to work under
RANDOMIZE_ALLOCATED_MEMORY.

Kevin Grittner sent in two revisions of a patch to fix an issue
manifesting as "multiple backends attempting to wait for pincount 1".

Kevin Grittner sent in a patch to help control bloat in the face of
old snapshots.

Peter Eisentraut sent in a patch to speed up make check-world.

Álvaro Herrera sent in another revision of a patch to add CREATE
ALTER, GRANT/REVOKE, COMMENT ON, and SECURITY LABEL command deparsing
support to event triggers.

Peter Eisentraut sent in a patch to add a pg_settings.pending_restart
column.

Peter Eisentraut sent in a patch to use forward slashes exclusively in
Windows builds, making it possible at least in theory to build such on
non-Windows machines.

Kevin Grittner sent in a patch to allow a "snapshot too old" error.

Andrew Dunstan sent in two revisions of a patch to add mogrify and
indent to JSONB.

Robert Haas sent in another revision of a patch to allow assessing
parallel safety.

Alexander Korotkov sent in another revision of a patch to implement
KNN-GiST with recheck.

Kaigai Kouhei sent in another revision of a patch to implement join
replacement with foreign-/custom-scan.

Petr (PJMODOS) Jelinek sent in another revision of a patch to
implement a sequence access method.

Tomas Vondra sent in a PoC patch to implement a mmap-based memory
context / allocator.

Emre Hasegeli sent in another revision of a patch to implement
selectivity estimation for inet operators.

Andres Freund sent in another revision of a patch to implement
replication identifiers.

David Steele sent in a patch to fix some dependency issues in the
documentation build for OSX.

Andrew (RhodiumToad) Gierth and Tomas Vondra traded patches to fix
some really bad blowups with hash outer join and nulls.