[ANNOUNCE] == PostgreSQL Weekly News - November 12 2017 ==

== PostgreSQL Weekly News - November 12 2017 ==

PostgreSQL security releases 10.1, 9.6.6, 9.5.10, 9.4.15, 9.3.20, and 9.2.24 are
out.  Please read the announcement below and upgrade at the next available
downtime.
https://www.postgresql.org/about/news/1801/

PGConf.Brazil 2018 will take place in São Paulo, Brazil on August 3-4 2018. The
CfP will open soon.
http://pgconf.com.br

Nordic PGDay 2018 will be held in Oslo, Norway, at the Radisson Blu Hotel
Nydalen, on March 13, 2018.  The CfP is open through December 31, 2017 at
https://2018.nordicpgday.org/cfp/

pgDay Paris 2018 will be held in Paris, France at the Espace Saint-Martin, on
March 15 2018.  The CfP is open until December 31, 2017.
http://2018.pgday.paris/callforpapers/

== PostgreSQL Jobs for November ==

http://archives.postgresql.org/pgsql-jobs/2017-11/

== PostgreSQL Local ==

PGConf Local: Seattle will be held November 13 - 14, 2017.
https://www.pgconf.us/#Seattle2017

PGDay Australia 2017 will be held on November 17 in Melbourne.
http://2017.pgday.com.au/

PostgreSQL Session will take place November 17th, 2017, in Paris, France.  
http://www.postgresql-sessions.org/en/9/start

PGDAY Brasilia 2017 will take place December 2, 2017.  The CfP
is open at 
http://www.pgdaybrasilia.com.br/submissao-de-palestra/

PGConf Local: Austin will be held December 4 - 5, 2017. Call for Papers is
now open at https://www.pgconf.us/conferences/Austin2017

PGConf.ASIA 2017 will take place on December 4-6 2017 in Akihabara, Tokyo,
Japan.
http://www.pgconf.asia/EN/2017/

FOSDEM PGDay 2018, a one day conference held before the main FOSDEM event will
be held in Brussels, Belgium, on Feb 2nd, 2018.
https://2018.fosdempgday.org/

Prague PostgreSQL Developer Day 2018 (P2D2 2018) is a two-day
conference that will be held on February 14-15 2018 in Prague, Czech Republic.
The CfP is open until January 5, 2018 at https://p2d2.cz/callforpapers
http://www.p2d2.cz/

PGConf India 2018 will be on February 22-23, 2018 in Bengaluru, Karnataka.
Proposals are due via https://goo.gl/forms/F9hRjOIsaNasVOAz2 by October 31st, 2017.
http://pgconf.in/

PostgreSQL@SCaLE is a two day, two track event which takes place on
March 8-9, 2018, at Pasadena Convention Center, as part of SCaLE 16X.
The CfP is open through October 31, 2017 at
http://www.socallinuxexpo.org/scale/16x/cfp

PGConf APAC 2018 will be held in Singapore March 22-24, 2018.  The CfP is open
at http://2018.pgconfapac.org/cfp through December 4, 2017.
http://2018.pgconfapac.org/

The German-speaking PostgreSQL Conference 2018 will take place on April 13th,
2018 in Berlin.  The CfP is open until January 09, 2018 at
http://2018.pgconf.de/de/callforpapers.html and the conference site is at
http://2018.pgconf.de/

PGCon 2018 will take place in Ottawa on May 29 - June 2018.  The CfP goes out
soon.
https://www.pgcon.org/2018/

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm EST5EDT.  Please send English
language ones to david@fetter.org, German language to pwn@pgug.de, Italian
language to pwn@itpug.org.

== Applied Patches ==

Tom Lane pushed:

- Release notes for 10.1, 9.6.6, 9.5.10, 9.4.15, 9.3.20, 9.2.24.  In the v10 branch, also back-patch the effects of
1ff01b390and c29c57890 on these files, to reduce future maintenance issues.  (I'd do it further back, except that the
9.Xbranches differ anyway due to xlog-to-wal link tag renaming.)
https://git.postgresql.org/pg/commitdiff/b35b185bf705c4dbaf21198c81b3d85f4a96804a

- Fix version numbering foulups exposed by 10.1.  configure computed PG_VERSION_NUM incorrectly.  (Coulda sworn I
testedthat logic back when, but it had an obvious thinko.) pg_upgrade had not been taught about the new dispensation
withjust one part in the major version number.  Both things accidentally failed to fail with 10.0, but with 10.1 we got
thewrong results. Per buildfarm. https://git.postgresql.org/pg/commitdiff/d0c80c17f1a6d0b93d2ca14fe47d83b131ce9108
 

- Fix unportable usage of <ctype.h> functions.  isdigit(), isspace(), etc are likely to give surprising results if
passeda signed char.  We should always cast the argument to unsigned char to avoid that.  Error in commit 63d6b97fd,
foundby buildfarm member gaur.  Back-patch to 9.3, like that commit.
https://git.postgresql.org/pg/commitdiff/d1f9ac5b100dbc4da02f0f209a2e7730bd5e83e9

- Fix unportable spelling of int64 constant.  Per buildfarm member pademelon.
https://git.postgresql.org/pg/commitdiff/92a1834dd88e56e823ac6641313a2f077a8af72e

- Fix two violations of the ResourceOwnerEnlarge/Remember protocol.  The point of having separate
ResourceOwnerEnlargeFooand ResourceOwnerRememberFoo functions is so that resource allocation can happen in between.
Doingit in some other order is just wrong.  OpenTemporaryFile() did open(), enlarge, remember, which would leak the
openfile if the enlarge step ran out of memory.  Because fd.c has its own layer of resource-remembering, the
consequenceslook like they'd be limited to an intratransaction FD leak, but it's still not good.  IncrBufferRefCount()
didenlarge, remember, incr-refcount, which would blow up if the incr-refcount step ever failed.  It was safe enough
whenwritten, but since the introduction of PrivateRefCountHash, I think the assumption that no error could happen there
ispretty shaky.  The odds of real problems from either bug are probably small, but still, back-patch to supported
branches. Thomas Munro and Tom Lane, per a comment from Andres Freund
https://git.postgresql.org/pg/commitdiff/c5269472ea9bb4a6fbb8a0510f7d676d725933ab

- Doc: fix erroneous example.  The grammar requires these options to appear the other way 'round.  jotpe@posteo.de
Discussion:https://postgr.es/m/78933bd0-45ce-690e-b832-a328dd1a5567@posteo.de
https://git.postgresql.org/pg/commitdiff/bd65e0c62486e6108a7dc824f918754a13072f7a

- Allow --with-bonjour to work with non-macOS implementations of Bonjour.  On macOS the relevant functions require no
speciallibrary, but elsewhere we need to pull in libdns_sd.  Back-patch to supported branches.  No docs change since
thedocs do not suggest that this is a Mac-only feature.  Luke Lonergan Discussion:
https://postgr.es/m/2D8331C5-D64F-44C1-8717-63EDC6EAF7EB@brightforge.com
https://git.postgresql.org/pg/commitdiff/9b9cb3c4534d717c1c95758670198ebbf8a20af2

- Last-minute updates for release notes.  Security: CVE-2017-12172, CVE-2017-15098, CVE-2017-15099
https://git.postgresql.org/pg/commitdiff/92d830f4bff643953a09563abaa106af42625207

- Add tests for json{b}_populate_recordset() crash case.  The problem reported as CVE-2017-15098 was already resolved
inHEAD by commit 37a795a60, but let's add the relevant test cases anyway.  Michael Paquier and Tom Lane, per a report
fromDavid Rowley.  Security: CVE-2017-15098
https://git.postgresql.org/pg/commitdiff/b574228715f0fd77ed1f4f084603cff9e757e992

- Revert "Allow --with-bonjour to work with non-macOS implementations of Bonjour.".  Upon further review, our Bonjour
codedoesn't actually work with the Avahi not-too-compatible compatibility library.  While you can get it to work on
non-macOSplatforms if you link to Apple's own mDNSResponder code, there don't seem to be many people who care about
that. Leaving in the AC_SEARCH_LIBS call seems more likely to encourage people to build broken configurations than to
doanything very useful.  Hence, remove the AC_SEARCH_LIBS call and put in a warning comment instead.  Discussion:
https://postgr.es/m/2D8331C5-D64F-44C1-8717-63EDC6EAF7EB@brightforge.com
https://git.postgresql.org/pg/commitdiff/20d9adab60754ac71b0b500c91c45e12e940b3ce

- Fix bogus logic for checking executables' versions within pg_upgrade. Somebody messed up a refactoring here.  As it
stood,we'd check pg_ctl's --version output twice for each cluster.  Worse, the first check for the new cluster's
versionhappened before we'd done any validate_exec checks there, breaking the check ordering the code intended.  A.
AkentevaDiscussion: https://postgr.es/m/f9266a85d918a3cf3a386b5148aee666@postgrespro.ru
https://git.postgresql.org/pg/commitdiff/9be95ef156e7c2ae0924300acddd483504fa33b3

- Fix typo in ALTER SYSTEM output.  The header comment written into postgresql.auto.conf by ALTER SYSTEM should match
whatinitdb put there originally.  Feike Steenbergen Discussion:
https://postgr.es/m/CAK_s-G0KcKdO=0hqZkwb3s+tqZuuHwWqmF5BDsmoO9FtX75r0g@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/6c3a7ba5bb0f960ed412b1c36e815f53347b3d79

- Restrict lo_import()/lo_export() via SQL permissions not hard-wired checks. While it's generally unwise to give
permissionson these functions to anyone but a superuser, we've been moving away from hard-wired permission checks
insidefunctions in favor of using the SQL permission system to control access.  Bring lo_import() and lo_export() into
compliancewith that approach. In particular, this removes the manual configuration option ALLOW_DANGEROUS_LO_FUNCTIONS.
That dates back to 1999 (commit 4cd4a54c8); it's unlikely anyone has used it in many years.  Moreover, if you really
wantsuch behavior, now you can get it with GRANT ... TO PUBLIC instead.  Michael Paquier Discussion:
https://postgr.es/m/CAB7nPqRHmNOYbETnc_2EjsuzSM00Z+BWKv9sy6tnvSd5gWT_JA@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/5ecc0d738e5864848bbc2d1d97e56d5846624ba2

- Refactor permissions checks for large objects.  Up to now, ACL checks for large objects happened at the level of the
SQL-callablefunctions, which led to CVE-2017-7548 because of a missing check.  Push them down to be enforced in
inv_api.cas much as possible, in hopes of preventing future bugs.  This does have the effect of moving read and write
permissionerrors to happen at lo_open time not loread or lowrite time, but that seems acceptable.  Michael Paquier and
TomLane Discussion: https://postgr.es/m/CAB7nPqRHmNOYbETnc_2EjsuzSM00Z+BWKv9sy6tnvSd5gWT_JA@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/ae20b23a9e7029f31ee902da08a464d968319f56

- Tighten test in contrib/bloom/t/001_wal.pl.  Make bloom WAL test compare psql output text, not just result codes;
thiswas evidently the intent all along, but it was mis-coded.  In passing, make sure we will notice any failure in
setupsteps.  Alexander Korotkov, reviewed by Michael Paquier and Masahiko Sawada Discussion:
https://postgr.es/m/CAPpHfdtohPdQ9rc5mdWjxq+3VsBNw534KV_5O65dTQrSdVJNgw@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/7e60e678615b1f803ac73faee71cca79ec310d1d

Noah Misch pushed:

- Add a temp-install prerequisite to "check"-like targets not having one. Makefile.global assigns this prerequisite to
everytarget named "check", but similar targets must mention it explicitly.  Affected targets failed, tested $PATH
binaries,or tested a stale temporary installation.  The src/test/modules examples worked properly when called as "make
-Csrc/test/modules/$FOO check", but "make -j" allowed the test to start before the temporary installation was in place.
Back-patch to 9.5, where commit dcae5faccab64776376d354decda0017c648bb53 introduced the shared temp-install.
https://git.postgresql.org/pg/commitdiff/c66b438db62748000700c9b90b585e756dd54141

- start-scripts: switch to $PGUSER before opening $PGLOG.  By default, $PGUSER has permission to unlink $PGLOG.  If
$PGUSERreplaces $PGLOG with a symbolic link, the server will corrupt the link-targeted file by appending log messages.
Sincethese scripts open $PGLOG as root, the attack works regardless of target file ownership.  "make install" does not
installthese scripts anywhere.  Users having manually installed them in the past should repeat that process to acquire
thisfix.  Most script users have $PGLOG writable to root only, located in $PGDATA.  Just before updating one of these
scripts,such users should rename $PGLOG to $PGLOG.old.  The script will then recreate $PGLOG with proper ownership.
Reviewedby Peter Eisentraut. Reported by Antoine Scemama.  Security: CVE-2017-12172
https://git.postgresql.org/pg/commitdiff/dfc015dcf46c1996bd7ed5866e9e045d258604b3

- Ignore XML declaration in xpath_internal(), for UTF8 databases.  When a value contained an XML declaration naming
someother encoding, this function interpreted UTF8 bytes as the named encoding, yielding mojibake.  xml_parse() already
hassimilar logic.  This would be necessary but not sufficient for non-UTF8 databases, so preserve behavior there until
thexpath facility can support such databases comprehensively.  Back-patch to 9.3 (all supported versions).  Pavel
Stehuleand Noah Misch Discussion:
https://postgr.es/m/CAFj8pRC-dM=tT=QkGi+Achkm+gwPmjyOayGuUfXVumCxkDgYWg@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/2918fcedbf2b2adab688a7849ecce4556ef912ac

- Fix previous commit's test, for non-UTF8 databases with non-XML builds.  To ensure stable output, catch one more
configuration-specificerror.  Back-patch to 9.3, like the commit that added the test.
https://git.postgresql.org/pg/commitdiff/4b865aee2582292a42a8e58247a41b46f5aa8a82

- Make connect/test1 independent of localhost IPv6.  Since commit 868898739a8da9ab74c105b8349b7b5c711f265a, it has
assumed"localhost" resolves to both ::1 and 127.0.0.1.  We gain nothing from that assumption, and it does not hold in a
defaultinstallation of Red Hat Enterprise Linux 5.  Back-patch to 9.3 (all supported versions).
https://git.postgresql.org/pg/commitdiff/34baf8a00b018caf7269134cf9b461266e66d9a7

- Add post-2010 ecpg tests to checktcp.  This suite had been a proper superset of the regular ecpg test suite, but the
threenewest tests didn't reach it. To make this less likely to recur, delete the extra schedule file and pass the
TCP-specifictest on the command line.  Back-patch to 9.3 (all supported versions).
https://git.postgresql.org/pg/commitdiff/0b7e76eb2b142d0b4a2a831e7fa1fac44820f52c

Simon Riggs pushed:

- Exclude pg_internal.init from BASE_BACKUP.  Add docs to explain this for other backup mechanisms Author: David Steele
<david@pgmasters.net>Reviewed-by: Petr Jelinek <petr.jelinek@2ndQuadrant.com> et al
https://git.postgresql.org/pg/commitdiff/98267ee83e068afa8bd85c8cc710ab2144e711fa

- Remove secondary checkpoint.  Previously server reserved WAL for last two checkpoints, which used too much disk space
forsmall servers.  Bumps PG_CONTROL_VERSION Author: Simon Riggs <simon@2ndQuadrant.com> Reviewed-by: Michael Paquier
<michael.paquier@gmail.com>https://git.postgresql.org/pg/commitdiff/4b0d28de06b28e57c540fca458e4853854fbeaf8
 

Robert Haas pushed:

- Fix typo in comment.  Masahiko Sawada Discussion:
http://postgr.es/m/CAD21AoDrf5AOpZ-mX-j6O=zFNFfKaTdHkv3o1X2eSs2nBXALug@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/698e5254377baffb95774d47872100997f135cba

- Fix incorrect comment.  Etsuro Fujita Discussion: http://postgr.es/m/5A05728E.4050009@lab.ntt.co.jp
https://git.postgresql.org/pg/commitdiff/b9941d3468505aea8bfdd74840b753ed27b9d29f

- Account for the effect of lossy pages when costing bitmap scans.  Dilip Kumar, reviewed by Alexander Kumenkov, Amul
Sul,and me.  Some final adjustments by me.  Discussion:
http://postgr.es/m/CAFiTN-sYtqUOXQ4SpuhTv0Z9gD0si3YxZGv_PQAAMX8qbOotcg@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/5edc63bda68a77c4d38f0cbeae1c4271f9ef4100

- Add hash partitioning.  Hash partitioning is useful when you want to partition a growing data set evenly.  This can
beuseful to keep table sizes reasonable, which makes maintenance operations such as VACUUM faster, or to enable
partition-wisejoin.  At present, we still depend on constraint exclusion for partitioning pruning, and the shape of the
partitionconstraints for hash partitioning is such that that doesn't work.  Work is underway to fix that, which should
bothimprove performance and make partitioning pruning work with hash partitioning.  Amul Sul, reviewed and tested by
DilipKumar, Ashutosh Bapat, Yugo Nagata, Rajkumar Raghuwanshi, Jesper Pedersen, and by me.  A few final tweaks also by
me. Discussion: http://postgr.es/m/CAAJ_b96fhpJAP=ALbETmeLk1Uni_GFZD938zgenhF49qgDTjaQ@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/1aba8e651ac3e37e1d2d875842de1e0ed22a651e

Peter Eisentraut pushed:

- Expand empty end tag. https://git.postgresql.org/pg/commitdiff/6e1e4c0d2f757677b03b6f164487a582c7dfab33

- Put markup in the right place. https://git.postgresql.org/pg/commitdiff/4497f2f3b30fa5cd48898033c351bfcf01ce73e2

- Change TRUE/FALSE to true/false.  The lower case spellings are C and C++ standard and are used in most parts of the
PostgreSQLsources.  The upper case spellings are only used in some files/modules.  So standardize on the standard
spellings. The APIs for ICU, Perl, and Windows define their own TRUE and FALSE, so those are left as is when using
thoseAPIs.  In code comments, we use the lower-case spelling for the C concepts and keep the upper-case spelling for
theSQL concepts.  Reviewed-by: Michael Paquier <michael.paquier@gmail.com>
https://git.postgresql.org/pg/commitdiff/2eb4a831e5fb5d8fc17e13aea56e04af3efe27b4

- Remove junk left from DSSSL to XSL conversion.
https://git.postgresql.org/pg/commitdiff/e7397f015c9589f95f5f5b48d7a274b2f1628971

- Add -wnet to SP invocations.  This causes a warning when accidentally backpatching an XML-style empty-element tag
like<xref linkend="abc"/>. https://git.postgresql.org/pg/commitdiff/9a8d3c4eeaf34966056a41a263c6e2ca4d5e4012
 

- Add some const decorations to prototypes.  Reviewed-by: Fabien COELHO <coelho@cri.ensmp.fr>
https://git.postgresql.org/pg/commitdiff/0e1539ba0d0a43de06c6e0572a565e73b9472538

- Fix some null pointer dereferences in LDAP auth code.  An LDAP URL without a host name such as "ldap://" or without a
baseDN such as "ldap://localhost" would cause a crash when reading pg_hba.conf.  If no binddn is configured, an error
messagemight end up trying to print a null pointer, which could crash on some platforms.  Author: Thomas Munro
<thomas.munro@enterprisedb.com>Reviewed-by: Michael Paquier <michael.paquier@gmail.com>
https://git.postgresql.org/pg/commitdiff/0c98d0dd5c85ce0c8485ae1a8351a26b83c4338b

Dean Rasheed pushed:

- Always require SELECT permission for ON CONFLICT DO UPDATE.  The update path of an INSERT ... ON CONFLICT DO UPDATE
requiresSELECT permission on the columns of the arbiter index, but it failed to check for that in the case of an
arbiterspecified by constraint name.  In addition, for a table with row level security enabled, it failed to check
updatedrows against the table's SELECT policies when the update path was taken (regardless of how the arbiter index was
specified). Backpatch to 9.5 where ON CONFLICT DO UPDATE and RLS were introduced.  Security: CVE-2017-15099
https://git.postgresql.org/pg/commitdiff/87b2ebd352c4afe1ded0841604b59a3afbae97d1

== Pending Patches ==

Edmund Horner sent in a patch to add some tab completion to the SELECT  target
list in psql.

Ashutosh Bapat sent in a patch to improve \d+ output of a partitioned table and
separate default partition from rest of the partitions in psql.

Noah Misch sent in a patch to fix a race to build pg_isolation_regress in "make
-j check-world".

David Rowley sent in a patch to remove useless DISTINCT clauses.

Thomas Munro sent in a patch atop the statement-level rollback patch to document
it better.

Konstantin Knizhnik sent in another revision of a patch to optimize secondary
index lookups.

Raúl Marín Rodríguez sent in three more revisions of a patch to add pow() to
pgbench.

Asim Praveen sent in two more revisions of a patch to change incorrect calls to
PageGetLSN to BufferGetLSNAtomic and assert that locks are properly held in
PageGetLSN.

Kyotaro HORIGUCHI and Pavel Stěhule traded patches to add default namespaces for
XPath expressions.

Amit Langote and David Rowley traded patches to prune partitions faster.

Haribabu Kommi sent in another revision of a patch to create the infrastructure
for pluggable storage.

Nathan Bossart sent in two more revisions of a patch to add new logging for
VACUUM and ANALYZE.

Haribabu Kommi sent in another revision of a patch to add a pg_stat_wal_write
statistics view.

Haribabu Kommi sent in two more revisions of a patch to refactor handling of
database attributes between pg_dump and pg_dumpall.

Tom Lane sent in a patch to speed up compactify_tuples, this time without a sort
step.

Alexander Korotkov sent in two more revisions of a patch to fix the bloom WAL
tap test.

Thomas Munro sent in a patch to add planning counters to pg_stat_statements.

Michaël Paquier and Fabrízio de Royes Mello traded patches to add session_start
and session_end hooks.

Dmitry Dolgov and Artur Zakirov traded patches to add generic subscripting,
implementing those for arrays and JSON with same.

Masahiko Sawada sent in a patch to remove a duplicate setting in
test/recovery/Makefile.

Kyotaro HORIGUCHI sent in another revision of a patch to restrict the maximum
keep segments by repslots.

Amit Khandekar sent in two more revisions of a patch to enable UPDATEs of a
partition key to move tuples to the appropriate partition.

Etsuro Fujita sent in a patch to reorder header files in alphabetical order.

Robert Haas sent in another revision of a patch to implement parallel append.

Amit Kapila sent in another revision of a patch to ensure that parallel paths
include tlist cost.

Beena Emerson sent in another revision of a patch to implement runtime partiton
pruning.

Pavel Stěhule sent in two more revisions of a patch to add \graw to psql.

Michaël Paquier sent in a patch to remove ALLOW_DANGEROUS_LO_FUNCTIONS for
LO-related superuser checks, replace superuser checks of large object
import/export by ACL checks, and move ACL checks for large objects when opening
them.

Huong Dangminh sent in a patch to fix an issue where user-defined data types
weren't working right under logical replication.

Mark Rofail sent in another revision of a patch to implement foreign key arrays.

David Rowley sent in a patch to add a mention of CREATE STATISTICS to the event
trigger docs.

Robert Haas sent in another revision of a patch to speed up processing at Gather
nodes.

Graham Leggett sent in a patch to log SSL certificate verification errors.

Robert Haas and Amit Kapila traded patches to parallelize queries containing
initplans.

Pavel Stěhule sent in another revision of a patch to add a SERVER_VERSION
variable to psql.

Robert Haas sent in a patch to ensure that GatherMerge pushes tlists down.

Thomas Munro sent in a patch to add a parallel_leader_participation GUC.

Andrey Borodin sent in another revision of a patch to add GiST VACUUM.



-- 
Sent via pgsql-announce mailing list (pgsql-announce@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-announce