== PostgreSQL Weekly News - August 22 2010 ==

== PostgreSQL Weekly News - August 22 2010 ==

Call for Papers for PGDay.EU 2010 held on December 6-8 in Stuttgart,
Germany is open.
http://2010.pgday.eu/callforpapers

== PostgreSQL Product News ==

pgpool-II 3.0beta1, a connection pooler and more, released.
http://pgfoundry.org/projects/pgpool/

== PostgreSQL Jobs for August ==

http://archives.postgresql.org/pgsql-jobs/2010-08/threads.php

== PostgreSQL Local ==

The Call for Papers for West is open until September 5, 2010.  Details at:
http://www.postgresqlconference.org/

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david@fetter.org, German language
to pwn@pgug.de, Italian language to pwn@itpug.org.  Spanish language
to pwn@arpug.com.ar.

== Applied Patches ==

Tom Lane committed:

- In pgsql/src/backend/utils/init/miscinit.c, arrange to fsync the
  contents of lockfiles (both postmaster.pid and the socket lockfile)
  when writing them.  The lack of an fsync here may well explain two
  different reports we've seen of corrupted lockfile contents, which
  doesn't particularly bother the running server but can prevent a new
  server from starting if the old one crashes.  Per suggestion from
  Alvaro Herrera.  Back-patch to all supported versions.

- Add missing handling of PlannedStmt.transientPlan in
  copyfuncs/outfuncs.  _outPlannedStmt is only debug support, so the
  omission there was not very serious, but the omission in
  _copyPlannedStmt is a real bug.  The consequence would be that a
  copied plan tree would never be marked as a transient plan, so that
  we would forget we ought to replan it after some not-yet-ready index
  becomes ready for use.  This might explain some past complaints
  about indexes created with CREATE INDEX CONCURRENTLY not being used
  right away.  Problem spotted by Yeb Havinga.  Back-patch to 8.3,
  where the field was added.

- Fix failure of "ALTER TABLE t ADD COLUMN c serial" when done by
  non-owner.  The implicitly created sequence was created as owned by
  the current user, who could be different from the table owner, eg if
  current user is a superuser or some member of the table's owning
  role.  This caused sanity checks in the SEQUENCE OWNED BY code to
  spit up.  Although possibly we don't need those sanity checks, the
  safest fix seems to be to make sure the implicit sequence is
  assigned the same owner role as the table has.  (We still do all
  permissions checks as the current user, however.) Per report from
  Josh Berkus.  Back-patch to 9.0.  The bug goes back to the invention
  of SEQUENCE OWNED BY in 8.2, but the fix requires an API change for
  DefineRelation(), which seems to have potential for breaking
  third-party code if done in a minor release.  Given the lack of
  prior complaints, it's probably not worth fixing in the stable
  branches.

- Rename utf2ucs() to utf8_to_unicode(), and export it so it can be
  used elsewhere.  Similarly rename the version in mbprint.c, not
  because this affects anything but just to keep the two copies in
  exact sync.  There was some discussion of having only one copy in
  src/port/ instead, but this function is so small and unlikely to
  change that that seems like overkill.  Slightly editorialized
  version of a patch by Joseph Adams.  (The bug-fix aspect of his
  patch was applied separately, and back-patched.)

- In pgsql/src/backend/executor/nodeModifyTable.c, reset the
  per-output-tuple exprcontext each time through the main loop in
  ExecModifyTable().  This avoids memory leakage when trigger
  functions leave junk behind in that context (as they more or less
  must).  Problem and solution identified by Dean Rasheed.  I'm a bit
  concerned about the longevity of this solution --- once a plan can
  have multiple ModifyTable nodes, we are very possibly going to have
  to do something different.  But it should hold up for 9.0.

- In pgsql/src/backend/commands/trigger.c, fix possible corruption of
  AfterTriggerEventLists in subtransaction rollback.
  afterTriggerInvokeEvents failed to adjust events->tailfree when
  truncating the last chunk of an event list.  This could result in
  the data being "de-truncated" by afterTriggerRestoreEventList during
  a subsequent subtransaction abort.  Even that wouldn't kill us,
  because the re-added data would just be events marked DONE ---
  unless the data had been partially overwritten by new events.  Then
  we might crash, or in any case misbehave (perhaps fire triggers
  twice, or fire triggers with the wrong event data).  Per bug #5622
  from Thue Janus Kristensen.  Back-patch to 8.4 where the current
  trigger list representation was introduced.

- In pgsql/src/backend/storage/buffer/localbuf.c, allocate local
  buffers in a context of their own, rather than dumping them into
  TopMemoryContext.  This makes no functional difference, but makes it
  easier to see what the space is being used for in MemoryContextStats
  dumps.  Per a recent example in which I was surprised by the size of
  TopMemoryContext.

- In pgsql/src/pl/plpgsql/src/pl_exec.c, be a bit less cavalier with
  both the code and the comment for UNKNOWN fix.

- In pgsql/src/pl/plpgsql/src/pl_exec.c, keep exec_simple_check_plan()
  from thinking "SELECT foo INTO bar" is simple.  It's not clear if
  this situation can occur in plpgsql other than via the EXECUTE USING
  case Heikki illustrated, which I will shortly close off.  However,
  ignoring the intoClause if it's there is surely wrong, so let's
  patch it for safety.  Backpatch to 8.3, which is as far back as this
  code has a PlannedStmt to deal with.  There might be another way to
  make an equivalent test before that, but since this is just
  preventing hypothetical bugs, I'm not going to obsess about it.

- In pgsql/src/pl/plpgsql/src/gram.y, allow USING and INTO clauses of
  plpgsql's EXECUTE to appear in either order.  Aside from being more
  forgiving, this prevents a rather surprising misbehavior when the
  "wrong" order was used: the old code didn't throw a syntax error,
  but absorbed the INTO clause into the last USING expression, which
  then did strange things downstream.  Intentionally not changing the
  documentation; we'll continue to advertise only the "standard"
  clause order.  Backpatch to 8.4, where the USING clause was added to
  EXECUTE.

- Bring some sanity to the trace_recovery_messages code and docs.  Per
  gripe from Fujii Masao, though this is not exactly his proposed
  patch.  Categorize as DEVELOPER_OPTIONS and set context PGC_SIGHUP,
  as per Fujii, but set the default to LOG because higher values
  aren't really sensible (see the code for trace_recovery()).  Fix the
  documentation to agree with the code and to try to explain what the
  variable actually does.  Get rid of no-op calls trace_recovery(LOG),
  which accomplish nothing except to demonstrate that this option
  confuses even its author.

- Avoid saying "random" when randomness is not actually meant.  Per
  Thom Brown.

- In pgsql/src/backend/parser/gram.y, add missing processing of
  OptTemp in CREATE IF NOT EXISTS variant for typed tables.  Noted by
  Robert Haas.

- In pgsql/src/bin/pg_dump/pg_backup_archiver.c, improve parallel
  restore's ability to cope with selective restore (-L option).  The
  original coding tended to break down in the face of modified restore
  orders, as shown in bug #5626 from Albert Ullrich, because it would
  flip over into parallel-restore operation too soon.  That causes
  problems because we don't have sufficient dependency information in
  dump archives to allow safe parallel processing of SECTION_PRE_DATA
  items.  Even if we did, it's probably undesirable to allow that to
  override the commanded restore order.  To fix the problem of omitted
  items causing unexpected changes in restore order, tweak
  SortTocFromFile so that omitted items end up at the head of the list
  not the tail.  This ensures that they'll be examined and their
  dependencies will be marked satisfied before we get to any
  interesting items.  In HEAD and 9.0, we can easily change
  restore_toc_entries_parallel so that all SECTION_PRE_DATA items are
  guaranteed to be processed in the initial serial-restore loop, and
  hence in commanded order.  Only DATA and POST_DATA items are
  candidates for parallel processing.  For them there might be
  variations from the commanded order because of parallelism, but we
  should do it in a safe order thanks to dependencies.  In 8.4 it's
  much harder to make such a guarantee.  I settled for not letting the
  initial loop break out into parallel processing mode if it sees a
  DATA/POST_DATA item that's not to be restored; this at least
  prevents a non-restorable item from causing premature exit from the
  loop.  This means that 8.4 will be more likely to fail given a
  badly-ordered -L list than 9.x, but we don't really promise any such
  thing will work anyway.

- In pgsql/src/backend/utils/adt/arrayfuncs.c, use a
  non-locale-dependent definition of isspace() in array_in/array_out.
  array_in discards unquoted leading and trailing whitespace in array
  values, while array_out is careful to quote array elements that
  contain whitespace.  This is problematic when the definition of
  "whitespace" varies between locales: array_in could drop characters
  that were meant to be part of the value.  To avoid that, lock down
  "whitespace" to mean only the traditional six ASCII space
  characters.  This change also works around a bug in OS X and some
  older BSD systems, in which isspace() could return true for
  character fragments in UTF8 locales.  (There may be other places in
  PG where that bug could cause problems, but this is the only one
  complained of so far; see recent report from Steven Schlansker.)
  Back-patch to 9.0, but not further.  Given the lack of previous
  reports of trouble, changing this behavior in stable branches seems
  to offer more risk of breaking applications than reward of avoiding
  problems.

Peter Eisentraut committed:

- Spell and markup checking for docs.

- In pgsql/doc/src/sgml/lo.sgml, revert: looks like Binary Large
  OBject[sic] wasn't a misspelling

- Backpatch some blatant spelling mistakes in the docs.

- Remove extra newlines at end and beginning of files, add missing
  newlines at end of files.

Michael Meskes committed:

- Applied Zoltan Boszormenyi's patch to fix a few memleaks in ecpg's
  pgtypeslib.

Heikki Linnakangas committed:

- In pgsql/src/backend/parser/parse_param.c, coerce 'unknown' type
  parameters to the right type in the fixed-params parse_analyze()
  function.  That case occurs e.g with PL/pgSQL EXECUTE ... USING
  'stringconstant'.  The coercion with a CoerceViaIO node.  The result
  is similar to the coercion via input function performed for unknown
  constants in coerce_type(), except that this happens at runtime.
  Backpatch to 9.0.  The issue is present in 8.4 as well, but the
  coerce param hook infrastructure this patch relies on was introduced
  in 9.0.  Given the lack of user reports and harmlessness of the bug,
  it's not worth attempting a different fix just for 8.4.

- Revert patch to coerce 'unknown' type parameters in the backend.  As
  Tom Lane pointed out, it would need a 2nd pass after the whole query
  is processed to correctly check that an unknown Param is coerced to
  the same target type everywhere.  Adding the 2nd pass would add a
  lot more code, which doesn't seem worth the risk given that there
  isn't much of a use case for passing unknown Params in the first
  place.  The code would work without that check, but it might be
  confusing and the behavior would be different from the varparams
  case.  Instead, just coerce all unknown params in a PL/pgSQL USING
  clause to text.  That's simple, and is usually what users expect.
  Revert the patch in CVS HEAD and master, and backpatch the new
  solution to 8.4. Unlike the previous solution, this applies easily
  to 8.4 too.

Robert Haas committed:

- Tidy up a few calls to smrgextend().  In the new API introduced by
  my patch to include the backend ID in temprel filenames, the last
  argument to smrgextend() became skipFsync rather than isTemp, but
  these calls didn't get the memo.  It's not really a problem to pass
  rel->rd_istemp rather than just plain false, because smgrextend()
  now automatically skips the fsync for temprels anyway, but this
  seems cleaner and saves some minute number of cycles.

- In pgsql/src/backend/storage/buffer/bufmgr.c, remove the isLocalBuf
  argument from ReadBuffer_common.  Since an SMgrRelation now knows
  whether or not the underlying relation is temporary, there's no
  point in also passing that information via an additional argument.

Magnus Hagander committed:

- Add vacuum and analyze counters to pg_stat_*_tables views.

- In pgsql/src/test/regress/expected/rules.out, adjust regression
  tests for previous commit, that I forgot to include...

Bruce Momjian committed:

- In pgsql/doc/src/sgml/config.sgml, document that autovacuum_freeze_max_age
  is used for pg_clog recycling.  We already mentioned xid wraparound.

== Rejected Patches (for now) ==

No one was disappointed this week :-)

== Pending Patches ==

Pavel Stehule sent in two revisions of a patch to extend psql's tab
completion to more entities.

KaiGai Kohei sent in a patch to implement preload libraries for
single-user mode.

Peter Eisentraut sent in a prototype query progress indicator.

Quan Zongliang sent in a patch to enable pg_ctl on Windows to select a
service start type (auto or on-demand), then a follow-up patch to add
the docs.  David Fetter unified these into standard format.

Robert Haas and Alex Hunsaker traded patches to create a new tool for
git histories.

Peter Eisentraut sent in a patch to fix an issue in PL/PythonU with
Python 2.7.

Heikki Linnakangas sent in a WIP patch to fix an issue with type
coercion in PL/pgsql's EXECUTE ... USING construct.

Pavel Stehule sent in two revisions of a patch to add MEDIAN() and
PERCENTILE() as contrib modules.

Boxuan Zhai sent in another revision of the MERGE patch.

Robert Haas sent in another refactor of comment.c.

Martin Pihlak sent in a patch to add a system-wide fallback directory
for SSL root certificates.

KaiGai Kohei sent in a patch to add a security hook at authorization
time.

Robert Haas sent in a patch to clean up smgr, which kicked off a
discussion of the design of the abstraction layers nearby.

Magnus Hagander sent in a patch to add columns the pg_stat_*_tables
which record the number of [auto]vacuum and [auto]analyze runs.

Peter Eisentraut sent in a patch to refactor makeVar in the back end.

Erik Rijkers sent in two revisions of a patch to make the debug
messages in pg_archivecleanup more consistent.

Magnus Hagander sent in a patch to track more vacuum stats.