== PostgreSQL Weekly News - November 27 2011 ==

== PostgreSQL Weekly News - November 27 2011 ==

FOSDEM 2012 - PostgreSQL Devroom: Call for Speakers is open until
December 20, 2011.
https://www.postgresql.eu/events/callforpapers/fosdem2012/

== PostgreSQL Jobs for November ==

http://archives.postgresql.org/pgsql-jobs/2011-11/threads.php

== PostgreSQL Local ==

The Call for Papers is open for PostgreSQL Session #3, which will be
held in Paris, Feb 2nd, 2012.  The deadline for proposals is the 30th
November 2011 and selected speakers will be notified by the 14th
December 2011.  Proposals (in French or English) should be submitted
to call-for-paper AT postgresql-sessions DOT org.
More information at: http://www.postgresql-sessions.org/en/3/

FOSDEM 2012 - PostgreSQL Devroom: Call for Speakers is open until
December 20, 2011.
https://www.postgresql.eu/events/callforpapers/fosdem2012/

The fifth annual "Prague PostgreSQL Developers Day" conference,
organized by CSPUG (Czech and Slovak PostgreSQL Users Group), will be
held on February 9, 2012 in Prague.  The Call for Papers is open.
Please send proposals including contact information, topic, and
expected length of session to info AT p2d2 DOT cz.

The Call for Papers for is open for FLOSS UK, which will be held in
Edinburgh from the 20th to the 22nd March 2012.  The deadline for
proposals is the 18th November 2011 and selected speakers will be
notified by the 25th November 2011.  Proposals should be submitted to
postgresql2012 AT flossuk DOT org.  More information at:
http://www.flossuk.org/Events/Spring2012


== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david@fetter.org, German language
to pwn@pgug.de, Italian language to pwn@itpug.org.  Spanish language
to pwn@arpug.com.ar.

== Reviews ==

== Applied Patches ==

Tom Lane pushed:

- Fix citext upgrade script to update derived copies of
  pg_type.typcollation.  If the existing citext type has not merely
  been created, but used in any tables, then the upgrade script wasn't
  doing enough.  We have to update attcollation for each citext table
  column, and indcollation for each citext index column, as well.  Per
  report from Rudolf van der Leeden.
  http://git.postgresql.org/pg/commitdiff/9b97b7f8356c63ea0b6704718d75ea01ec3035bf

- More code review for rangetypes patch.  Fix up some infelicitous
  coding in DefineRange, and add some missing error checks.  Rearrange
  operator strategy number assignments for GiST anyrange opclass so
  that they don't make such a mess of opr_sanity's table of operator
  names associated with different strategy numbers.  Assign
  hopefully-temporary selectivity estimators to range operators that
  didn't have one --- poor as the estimates are, they're still a lot
  better than the default 0.5 estimate, and they'll shut up the
  opr_sanity test that wants to see selectivity estimators on all
  built-in operators.
  http://git.postgresql.org/pg/commitdiff/a4ffcc8e115ed637f69ecb0295d78cc97f08a483

- Still more review for range-types patch.  Per discussion, relax the
  range input/construction rules so that the only hard error is lower
  bound > upper bound.  Cases where the lower bound is <= upper bound,
  but the range nonetheless normalizes to empty, are now permitted.
  Fix core dump in range_adjacent when bounds are infinite.  Marginal
  cleanup of regression test cases, some more code commenting.
  http://git.postgresql.org/pg/commitdiff/766948beddef66dd89563f465919eca6e131861c

- Improve implementation of range-contains-element tests.  Implement
  these tests directly instead of constructing a singleton range and
  then applying range-contains.  This saves a range
  serialize/deserialize cycle as well as a couple of redundant
  bound-comparison steps, and adds very little code on net.  Remove
  elem_contained_by_range from the GiST opclass: it doesn't belong
  there because there is no way to use it in an index clause (where
  the indexed column would have to be on the left).  Its commutator is
  in the opclass, and that's what counts.
  http://git.postgresql.org/pg/commitdiff/cddc819e45010492da00164d225a749661f43aef

- Remove zero- and one-argument range constructor functions.  Per
  discussion, the zero-argument forms aren't really worth the catalog
  space (just write 'empty' instead).  The one-argument forms have
  some use, but they also have a serious problem with looking too much
  like functional cast notation; to the point where in many real
  use-cases, the parser would misinterpret what was wanted.
  Committing this as a separate patch, with the thought that we might
  want to revert part or all of it if we can think of some way around
  the cast ambiguity.
  http://git.postgresql.org/pg/commitdiff/df73584431e7edb1dd76578777bd0fcc17b916a1

- Remove user-selectable ANALYZE option for range types.  It's not
  clear that a per-datatype typanalyze function would be any more
  useful than a generic typanalyze for ranges.  What *is* clear is
  that letting unprivileged users select typanalyze functions is a
  crash risk or worse.  So remove the option from CREATE TYPE AS
  RANGE, and instead put in a generic typanalyze function for ranges.
  The generic function does nothing as yet, but hopefully we'll
  improve that before 9.2 release.
  http://git.postgresql.org/pg/commitdiff/74c1723fc8dca2d70576ef2f0a66f4a7c99c173a

- Creator of a range type must have permission to call support
  functions.  Since range types can be created by non-superusers, we
  need to consider their permissions.  Ideally we'd check this when
  the type is used, not when it's created, but that seems like much
  more trouble than it's worth.  The existing restriction that the
  support functions be immutable already prevents most cases where an
  unauthorized call to a function might be thought a security issue,
  and the fact that the user has no access to the results of the
  system's calls to subtype_diff closes off the other plausible reason
  for concern.  So this check is basically pro-forma, but let's make
  it anyway.
  http://git.postgresql.org/pg/commitdiff/a912a2784be5d144aab89e447dfe8ca74b6ad079

- Adjust range_adjacent to support different canonicalization rules.
  The original coding would not work for discrete ranges in which the
  canonicalization rule is to produce symmetric boundaries (either []
  or () style), as noted by Jeff Davis.  Florian Pflug pointed out
  that we could fix that by invoking the canonicalization function to
  see if the range "between" the two given ranges normalizes to empty.
  This implementation of Florian's idea is a tad slower than the
  original code, but only in the case where there actually is a
  canonicalization function --- if not, it's essentially the same
  logic as before.
  http://git.postgresql.org/pg/commitdiff/b7056b832444696c931d59af057b0a345f5ae178

- Some more editing of the range-types documentation.  Be more
  thorough about specifying the expectations for canonical and
  subtype_diff functions, and move that info to the same place.
  http://git.postgresql.org/pg/commitdiff/604d4c4c95c44090af25083ce6624fea3ebb4553

- Fix unsupported options in CREATE TABLE ... AS EXECUTE.  The WITH
  [NO] DATA option was not supported, nor the ability to specify
  replacement column names; the former limitation wasn't even
  documented, as per recent complaint from Naoya Anzai.  Fix by moving
  the responsibility for supporting these options into the executor.
  It actually takes less code this way ...  catversion bump due to
  change in representation of IntoClause, which might affect stored
  rules.
  http://git.postgresql.org/pg/commitdiff/9ed439a9c07b69c2617cc98596611fdbdc22472c

- Fix erroneous replay of GIN_UPDATE_META_PAGE WAL records.  A simple
  thinko in ginRedoUpdateMetapage, namely failing to increment a loop
  counter, led to inserting records into the last pending-list page in
  the wrong order (the opposite of that intended).  So far as I can
  tell, this would not upset the code that eventually flushes pending
  items into the main part of the GIN index.  But it did break the
  code that searched the pending list for matches, resulting in
  transient failure to find matching entries during index lookups, as
  illustrated in bug #6307 from Maksym Boguk.  Back-patch to 8.4 where
  the incorrect code was introduced.
  http://git.postgresql.org/pg/commitdiff/877b67c38b946dcbf70fe11736bdde841e4c826b

- Fix overly-aggressive and inconsistent quoting in OS X start script.
  Sidar Lopez, per bug #6310, with some additional improvements by me.
  Back-patch to 9.0, where the issue was introduced.
  http://git.postgresql.org/pg/commitdiff/6c8768c3861d6690656b74676c44ffa63c0e4ef7

- Make GiST index searches smarter about queries against empty ranges.
  In the cases where the result of the called proc is negated, we
  should explicitly test both inputs for empty, to ensure we'll never
  return "true" for an unsatisfiable query.  In other cases we can
  rely on the called proc to say the right thing.
  http://git.postgresql.org/pg/commitdiff/5966bcecf6167f2921e614e66499fa4d2c195c64

- Use the proper macro to convert a bool to a Datum.  The original
  coding was var->value = (Datum) state; which is bogus, and then in
  commit 2f0f7b4bce13e68394543728801ef011fd82fac6 it was "corrected"
  to var->value = PointerGetDatum(state); which is a faithful
  translation but still wrong.  This seems purely cosmetic, though, so
  no need for a back-patch.  Pavel Stehule
  http://git.postgresql.org/pg/commitdiff/8722a1a06aedbbbeb4f848a7b9ee62d6ae8649c6

- Improve GiST range-contained-by searches by adding a flag for empty
  ranges.  In the original implementation, a range-contained-by search
  had to scan the entire index because an empty range could be lurking
  anywhere.  Improve that by adding a flag to upper GiST entries that
  says whether the represented subtree contains any empty ranges.
  Also, make a simple mod to the penalty function to discourage empty
  ranges from getting pushed into subtrees without any.  This needs
  more work, and the picksplit function should be taught about it too,
  but that code can be improved without causing an on-disk
  compatibility break; so we'll leave it for another day.  Since we're
  breaking on-disk compatibility of range values anyway, I took the
  opportunity to reorganize the range flags bits; the unused
  RANGE_xB_NULL bits are now adjacent, which might open the door for
  using them in some other way later.  In passing, remove the GiST
  range opclass entry for <>, which doesn't seem like it can really be
  indexed usefully.  Alexander Korotkov, with some editorializing by
  Tom Lane.
  http://git.postgresql.org/pg/commitdiff/c66e4f138b04d749a713ad075e16f3d60975f5ad

- Use IEEE infinity, not 1e10, for null-and-not-null case in
  gistpenalty().  Use of a randomly chosen large value was never
  exactly graceful, and now that there are penalty functions that are
  intentionally using infinity, it doesn't seem like a good idea for
  null-vs-not-null to be using something less.
  http://git.postgresql.org/pg/commitdiff/9f4563f743eab0682f908d51fa3a9c630b31322d

- Ensure that whole-row junk Vars are always of composite type.  The
  EvalPlanQual machinery assumes that whole-row Vars generated for the
  outputs of non-table RTEs will be of composite types.  However, for
  the case where the RTE is a function call returning a scalar type,
  we were doing the wrong thing, as a result of sharing code with a
  parser case where the function's scalar output is wanted.  (Or at
  least, that's what that case has done historically; it does seem a
  bit inconsistent.) To fix, extend makeWholeRowVar's API so that it
  can support both use-cases.  This fixes Belinda Cussen's report of
  crashes during concurrent execution of UPDATEs involving joins to
  the result of UNNEST() --- in READ COMMITTED mode, we'd run the
  EvalPlanQual machinery after a conflicting row update commits, and
  it was expecting to get a HeapTuple not a scalar datum from the
  "wholerowN" variable referencing the function RTE.  Back-patch to
  9.0 where the current EvalPlanQual implementation appeared.  In 9.1
  and up, this patch also fixes failure to attach the correct
  collation to the Var generated for a scalar-result case.  An
  example: regression=# select upper(x.*) from textcat('ab', 'cd') x;
  ERROR:  could not determine which collation to use for upper()
  function
  http://git.postgresql.org/pg/commitdiff/dd3bab5fd74db009c946278bb314c8458a2fef11

Simon Riggs pushed:

- Continue to allow VACUUM to mark last block of index dirty even when
  there is no work to do. Further analysis required.  Revert of patch
  c1458cc495ff800cd176a1c2e56d8b62680d9b71
  http://git.postgresql.org/pg/commitdiff/2d2841a56c8fa37a5dd5c6d33488ba6ca37116ff

Peter Eisentraut pushed:

- Small markup and wording improvement
  http://git.postgresql.org/pg/commitdiff/024ea25ccd553aa0ecb41d626a1016470e25a9a0

- Add pg_upgrade test suite.  It runs the regression tests, runs
  pg_upgrade on the populated database, and compares the before and
  after dumps.  While not actually a cross-version upgrade, this does
  detect omissions and bugs in the involved tools from time to time.
  It's also possible to do a cross-version upgrade by manually
  supplying parameters.
  http://git.postgresql.org/pg/commitdiff/08da2d282f1c3cbff141ecd218d737990cf6d234

Robert Haas pushed:

- Check for INSERT privileges in SELECT INTO / CREATE TABLE AS.  In
  the normal course of events, this matters only if ALTER DEFAULT
  PRIVILEGES has been used to revoke default INSERT permission.
  Whether or not the new behavior is more or less likely to be what
  the user wants when dealing only with the built-in privilege
  facilities is arguable, but it's clearly better when using a
  loadable module such as sepgsql that may use the hook in
  ExecCheckRTPerms to enforce additional permissions checks.  KaiGai
  Kohei, reviewed by Laurenz Albe
  http://git.postgresql.org/pg/commitdiff/f1b4aa2a84732255bd8a34fc9c7994a04409b77a

- Move "hot" members of PGPROC into a separate PGXACT array.  This
  speeds up snapshot-taking and reduces ProcArrayLock contention.
  Also, the PGPROC (and PGXACT) structures used by two-phase commit
  are now allocated as part of the main array, rather than in a
  separate array, and we keep ProcArray sorted in pointer order.
  These changes are intended to minimize the number of cache lines
  that must be pulled in to take a snapshot, and testing shows a
  substantial increase in performance on both read and write workloads
  at high concurrencies.  Pavan Deolasee, Heikki Linnakangas, Robert
  Haas
  http://git.postgresql.org/pg/commitdiff/ed0b409d22346b1b027a4c2099ca66984d94b6dd

Bruce Momjian pushed:

- Add pg_upgrade ENABLE_SAME_CATVERSION_UPGRADES macro for testing to
  allow upgrades of the same catalog version.  (Doesn't work for
  tablespaces, as indicated by C comment.)
  http://git.postgresql.org/pg/commitdiff/5df1403b0f2b44235c8f401bd49dab9a8cf6bf90

- Allow pg_upgrade to upgrade clusters that use exclusion contraints
  by fixing pg_dump to properly preserve such indexes.  Backpatch to
  9.1 and 9.0 (where the bug was introduced).
  http://git.postgresql.org/pg/commitdiff/35e27226b6751a90c9c911d08cb47b9f8a58825f

- Modify pg_dump to use error-free memory allocation macros.  This
  avoids ignoring errors and call-site error checking.
  http://git.postgresql.org/pg/commitdiff/3c0afde11a12bb3ca7c68a30ad0dedaa0d1adef5

- Remove ENABLE_SAME_CATVERSION_UPGRADES as unnecessary.
  http://git.postgresql.org/pg/commitdiff/32fb4546e3f8cf9c2ef524e9c20766a046521193

- Fix join_path_components() to not add a leading slash when joining
  to an initial null string.  Per report from Robert Haas in testing
  psql \ir.
  http://git.postgresql.org/pg/commitdiff/fd6dbc24ef1dc6b39a795b5e0e959cf500ad71d6

- Move pg_dump memory routines into pg_dumpmem.c/h and restore
  common.c with its original functions.  The previous function
  migration would cause too many difficulties in back-patching.
  http://git.postgresql.org/pg/commitdiff/9a7d49d1fba666c8bfb9df0065545e85d54dbc41

Heikki Linnakangas pushed:

- Preserve SQLSTATE when an SPI error is propagated through PL/python
  exception handler. This was a regression in 9.1, when the capability
  to catch specific SPI errors was added, so backpatch to 9.1.  Mika
  Eloranta, with some editing by Jan Urbański.
  http://git.postgresql.org/pg/commitdiff/f21fc7f9fc63ff86d7d77d352ae274b6e2b6e09e

- Take fillfactor into account in the new COPY bulk heap insert code.
  Jeff Janes
  http://git.postgresql.org/pg/commitdiff/dea5f6cefe74a6cb51a1cfd39e12a2e1a026df35

Alvaro Herrera pushed:

- Dump an unvalidated constraint separately from its table.  This
  allows possibly violating data to be imported before the constraint
  is installed.  Bug reported by Thom Brown
  http://git.postgresql.org/pg/commitdiff/e90710f34a6d5a73475e4241852b377675600325

- Improve logging of autovacuum I/O activity.  This adds some I/O
  stats to the logging of autovacuum (when the operation takes long
  enough that log_autovacuum_min_duration causes it to be logged), so
  that it is easier to tune.  Notably, it adds buffer I/O counts
  (hits, misses, dirtied) and read and write rate.  Authors: Greg
  Smith and Noah Misch
  http://git.postgresql.org/pg/commitdiff/9d3b50244357ef4c4e3b6e01f91de599077179c8

- Fix unvalidated check constraints on domains, too.  Same bug as
  reported by Thom Brown for check constraints on tables: the
  constraint must be dumped separately from the domain, otherwise it
  is restored before the data and thus prevents potentially-violating
  data from being loaded in the first place.  Per Dean Rasheed
  http://git.postgresql.org/pg/commitdiff/f717f4bca298b0a1170ff506e02142e4859c3dae

Andrew Dunstan pushed:

- Ensure plperl strings are always correctly UTF8 encoded.  Amit
  Khandekar and Alex Hunsaker.  Backpatched to 9.1 where the problem
  first occurred.
  http://git.postgresql.org/pg/commitdiff/4cdb41b54e432ee75e3c61e990b735b15fa66e81

- Use the preferred version of xsubpp, not necessarily the one that
  came with the distro version of perl.  David Wheeler and Alex
  Hunsaker.  Backpatch to 9.1 where it applies cleanly. A simple
  workaround is available for earlier branches, and further effort
  doesn't seem warranted.
  http://git.postgresql.org/pg/commitdiff/ba00ab0b111a0cbbac612e8ea8b0d5f96534102e

- Use the right interpreter for encoding test.
  http://git.postgresql.org/pg/commitdiff/efb0423cc7c6e46c22816e2acc48e8f827e341ed

- Fix MSVC builds broken by xsubpp change
  http://git.postgresql.org/pg/commitdiff/4a9018135a37203e176a561913453b38788ac7da

- Make pg_dumpall build with the right object files under MSVC.  This
  fixes a longstanding but up to now benign bug in the way pg_dumpall
  was built. The bug was exposed by recent code adjustments. The
  Makefile does not use $(OBJS) to build pg_dumpall, so this fix
  removes their source files from the pg_dumpall object and adds in
  the one source file it consequently needs.
  http://git.postgresql.org/pg/commitdiff/91572ee0a6dfeb62dda6c375f613d1b7fdfc1383

== Rejected Patches (for now) ==

No one was disappointed this week :-)

== Pending Patches ==

Mark Kirkwood sent in two revisions of a patch to allow renaming a
database which has backends connected to it.

Peter Geoghegan sent in four more revisions of a patch to inline
comparators as a performance optimization.

Alexander Korotkov sent in a WIP patch to allow index support for
regex operators.

Jan Urbanski sent in another revision of the patch to add cursor
support to PL/PythonU.

Pavel Stehule sent in a WIP patch to enable better support for
debugging overloaded functions in PL/pgsql.

Lars Kanis sent in two revisions of a patch to fix some infelicities
in certain versions of MSVC.

Pavel Stehule sent in a PoC patch to use errcontext for custom
exceptions in PL/pgsql.

Andrew Dunstan sent in another revision of a patch to add a \setenv
command to psql.

Dimitri Fontaine sent in another revision of the patch to add command
triggers.

Peter Eisentraut sent in a patch to fix error reports in vpath builds.

Pavel Stehule sent in another revision of the patch to add CHECK
FUNCTION and CHECK TRIGGER functionality.

Andres Freund and Pavan Deolasee traded patches to avoid unneeded
computation of snapshots.

Peter Eisentraut sent in a patch to allow psql to report the line
number on which an error occurred when reading from stdin.

Ants Aasma sent in a patch to implement timing of shared buffer fills
and per relation stats collection of said timings.  Buffer flushes are
timed as well but aren't exposed per table because of difficulty of
correctly attributing them.