== PostgreSQL Weekly News - April 05 2009 ==

== PostgreSQL Weekly News - April 05 2009 ==

Registration is now open for pgCon in Ottawa:
http://www.pgcon.org/2009/registration.php

Registration is now open for OSCON.  There will be a pgDay the Sunday
before OSCON: https://en.oreilly.com/oscon2009/public/register

PgDay Florianopolis will be May 22. Contact Dickson S. Guedes (guedes
AT guedesoft DOT net) to participate or submit a paper.
http://www.postgresql.org.br/eventos/pgday/sc

== PostgreSQL Product News ==

Archiveopteryx 3.0.7 and 3.1.0, a PostgreSQL-based email handler, released.
http://www.archiveopteryx.org/3.0.7
http://www.archiveopteryx.org/3.1.0

PGORM 0.09, a .NET object-relational mapper for PostgreSQL, released.
http://pgorm.googlecode.com

PostgreDAC ver. 2.5.1 released.
http://microolap.com/products/connectivity/postgresdac/download/

== PostgreSQL 8.4 Feature of the Week ==

- CIText: David Wheeler added a new contrib module which implements a
  full-featured Case Insensitive Text datatype.

== PostgreSQL Tip of the Week ==

- \h <COMMAND> in psql will give you the full documentation on a large
  variety of SQL commands.

== PostgreSQL Jobs for April ==

http://archives.postgresql.org/pgsql-jobs/2009-04/threads.php

== PostgreSQL Local ==

Kevin Kempter will be speaking about backup and recover at Denver
PUG's first meeting on April 8, 2009.
http://www.diapug.org/Meeting.html

PgDay Brasilia will be April 17.  Contact fernando.ike AT gmail DOT
com or eduardo DOT santos AT planejamento DOT gov DOT br to
participate.
http://www.postgresql.org.br/drupal6/eventos/pgday/df

Percona Performance Conference will be taking place at the Santa Clara
Convention Center, Santa Clara, California USA
http://conferences.percona.com/

CfP is open for PgDay Sao Paulo, which will be April 24.  Contact
marins DOT consultoria AT gmail DOT com or marcelojscosta AT gmail DOT
com to participate.

PostgreSQL Conference, U.S. (JDCon) will be holding a PgDay at
LinuxFest Northwest (April 25/26th). The call for papers is out at
http://www.postgresqlconference.org/

There will also be PgDays on April 29 in Porto Velho, RO and on April
30 in Ji-Parana, RO.  Contact Luis Fernando Bueno: proflfbueno AT
gmail DOT com to participate.

Michael Renner will be giving a PostgreSQL replication workshop at
Netways OSDC 2009 on April 29 and 30 in Nuremberg, Germany.
http://www.netways.de/english/osdc/y2009/programm/w/michael_renner_postgresql_repliziert_ein_ueberblick/

PGCon 2009 will be held 21-22 May 2009, in Ottawa at the University of
Ottawa.  It will be preceded by two days of tutorials on 19-20 May
2009.
http://www.pgcon.org/2009/

Save The Date: pgDay San Jose.  Sunday, July 19th 2009 immediately
before OSCON.  CfP, more info TBA!

PGCon Brazil will be take place October 23-24 2009 at Unicamp in
Campinas, Sao Paulo state.

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter
and Josh Berkus.

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david@fetter.org, German language
to pwn@pgug.de, Italian language to pwn@itpug.org.

== Applied Patches ==

Tom Lane committed:

- Fix an oversight in the support for storing/retrieving "minimal
  tuples" in TupleTableSlots.  We have functions for retrieving a
  minimal tuple from a slot after storing a regular tuple in it, or
  vice versa; but these were implemented by converting the internal
  storage from one format to the other.  The problem with that is it
  invalidates any pass-by-reference Datums that were already fetched
  from the slot, since they'll be pointing into the just-freed version
  of the tuple.  The known problem cases involve fetching both a
  whole-row variable and a pass-by-reference value from a slot that is
  fed from a tuplestore or tuplesort object.  The added regression
  tests illustrate some simple cases, but there may be other failure
  scenarios traceable to the same bug.  Note that the added tests
  probably only fail on unpatched code if it's built with
  --enable-cassert; otherwise the bug leads to fetching from freed
  memory, which will not have been overwritten without additional
  conditions.  Fix by allowing a slot to contain both formats
  simultaneously; which turns out not to complicate the logic much at
  all, if anything it seems less contorted than before.  Back-patch to
  8.2, where minimal tuples were introduced.

- In pgsql/src/backend/optimizer/plan/planner.c, fix window function
  plan generation to cope with volatile sort expressions.  (Not clear
  how useful these really are, but failing is no good...) Per report
  from David Fetter and Robert Treat.

- Add PQinitOpenSSL() function to support applications that use
  libcrypto but not OpenSSL (or perhaps vice versa, if that's
  possible).  Andrew Chernow, with minor editorialization by me.

- Add a "relistemp" boolean column to pg_class, which is true for
  temporary relations (including a temp table's indexes and toast
  table/index), and false for normal relations.  For ease of checking,
  this commit just adds the column and fills it correctly --- revising
  the relation access machinery to use it will come separately.

- Modify the relcache to record the temp status of both local and
  nonlocal temp relations; this is no more expensive than before, now
  that we have pg_class.relistemp.  Insert tests into bufmgr.c to
  prevent attempting to fetch pages from nonlocal temp relations.
  This provides a low-level defense against bugs-of-omission allowing
  temp pages to be loaded into shared buffers, as in the
  contrib/pgstattuple problem reported by Stuart Bishop.  While at it,
  tweak a bunch of places to use new relcache tests (instead of
  expensive probes into pg_namespace) to detect local or nonlocal temp
  tables.

- Fix contrib/pgstattuple and contrib/pageinspect to prevent attempts
  to read temporary tables of other sessions; that is unsafe because
  of the way our buffer management works.  Per report from Stuart
  Bishop.  This is redundant with the bufmgr.c checks in HEAD, but not
  at all redundant in the back branches.

- Remove last references to the crypt auth method, per Andreas 'ads'
  Scherbaum.

- In pgsql/doc/src/sgml/func.sgml, index some array functions, per
  Mario Splivalo.

- In pgsql/src/bin/pg_dump/pg_dump.c, improve pg_dump's query for
  retrieving BLOB comments to be more efficient when there are many
  blobs and not so many comments.  Tamas Vincze.

- In pgsql/src/pl/plpgsql/src/pl_exec.c, plpgsql's exec_simple_cast_value()
  mistakenly supposed that it could bypass casting effort whenever the
  input value was NULL.  However this prevents application of not-null
  domain constraints in the cases that use this function, as
  illustrated in bug #4741.  Since this function isn't meant for use
  in performance-critical paths anyway, this certainly seems like
  another case of "premature optimization is the root of all evil".
  Back-patch as far as 8.2; older versions made no effort to enforce
  domain constraints here anyway.

- In pgsql/src/backend/utils/misc/guc.c, fix GUC's reports of
  assign_hook failure to always include the parameter value we failed
  to assign, even in "can't happen" cases.  Motivated by wondering
  what's going on in a recent trouble report where "failed to commit"
  did happen.

- Fix SetClientEncoding() to maintain a cache of previously selected
  encoding conversion functions.  This allows transaction rollback to
  revert to a previous client_encoding setting without doing fresh
  catalog lookups.  I believe that this explains and fixes the recent
  report of "failed to commit client_encoding" failures.  This bug is
  present in 8.3.x, but it doesn't seem prudent to back-patch the fix,
  at least not till it's had some time for field testing in HEAD.  In
  passing, remove SetDefaultClientEncoding(), which was used nowhere.

- In pgsql/src/pl/plpgsql/src/pl_exec.c, minor code
  beautification/consolidation.

- Refactor ExecProject and associated routines so that fast-path code
  is used for simple Var targetlist entries all the time, even when
  there are other entries that are not simple Vars.  Also, ensure that
  we prefetch attributes (with slot_getsomeattrs) for all Vars in the
  targetlist, even those buried within expressions.  In combination
  these changes seem to significantly reduce the runtime for cases
  where tlists are mostly but not exclusively Vars.  Per my proposal
  of yesterday.

- Defend against possible crash if a plpython function does not
  specify names for its arguments.  Also add a regression test, since
  someone apparently changed every single plpython test case to use
  only named parameters; else we'd have noticed this sooner.  Euler
  Taveira de Oliveira, per a report from Alvaro Herrera.

- In pgsql/src/backend/storage/buffer/bufmgr.c, add a comment
  documenting the question of whether PrefetchBuffer should try to
  protect an already-existing buffer from being evicted.  This was
  left as an open issue when the posix_fadvise patch was committed.
  I'm not sure there's any evidence to justify more work in this area,
  but we should have some record about it in the source code.

- In pgsql/src/port/path.c, use (unsigned char) cast in argument of
  pg_tolower().  Maybe it works on Windows without that, but we
  shouldn't put bad examples where people might copy them.  Also,
  reformat slightly to improve the odds that pgindent won't go nuts on
  this.

- In pgsql/src/bin/psql/help.c, add missing help output for \ef
  option.  Andrew (RhodiumToad) Gierth.

- In pgsql/doc/src/sgml/ref/psql-ref.sgml, minor wordsmithing on
  descriptions of some \d commands.

- In pgsql/src/bin/psql/describe.c, make \dt \di and friends more
  consistent about the treatment of TOAST tables and indexes; to wit,
  never show either.  (You can examine them with plain \d if you're
  really so inclined.)

- In pgsql/src/bin/psql/describe.c, improve obsolete comment.

- Rewrite interval_hash() so that the hashcodes are equal for values
  that interval_eq() considers equal.  I'm not sure how that
  fundamental requirement escaped us through multiple revisions of
  this hash function, but there it is; it's been wrong since
  interval_hash was first written for PG 7.1.  Per bug #4748 from
  Roman Kononov.  Backpatch to all supported releases.  This patch
  changes the contents of hash indexes for interval columns.  That's
  no particular problem for PG 8.4, since we've broken on-disk
  compatibility of hash indexes already; but it will require a
  migration warning note in the next minor releases of all existing
  branches: "if you have any hash indexes on columns of type interval,
  REINDEX them after updating".

- A session that does not have any live snapshots does not have to be
  waited for when we are waiting for old snapshots to go away during a
  concurrent index build.  In particular, this rule lets us avoid
  waiting for idle-in-transaction sessions.  This logic could be
  improved further if we had some way to wake up when the session we
  are currently waiting for goes idle-in-transaction.  However that
  would be a significantly more complex/invasive patch, so it'll have
  to wait for some other day.  Simon Riggs, with some improvements by
  Tom.

- Remove the recently added node types ReloptElem and OptionDefElem in
  favor of adding optional namespace and action fields to DefElem.
  Having three node types that do essentially the same thing bloats
  the code and leads to errors of confusion, such as in yesterday's
  bug report from Khee Chin.

- Make an attempt at fixing our current Solaris 11 breakage: add a
  configure probe for opterr (exactly like the one for optreset) and
  have getopt.c define the variables only if configure doesn't find
  them in libc.

- In pgsql/src/port/getopt.c, hmm, baiji thinks we need explicit
  'extern' here.

- Remove contrib/intarray's definitions of the <@ and @> operators, so
  that they don't cause confusion with the built-in anyarray versions
  of those operators.  Adjust the module's index opclasses to support
  the built-in operators in place of the private ones.  The private
  implementations are still available under their historical names @
  and ~, so no functionality is lost.  Some quick testing suggests
  that they offer no real benefit over the core operators, however.
  Per a complaint from Rusty Conover.

- In pgsql/src/include/pg_config.h.win32, I had always wondered why
  pg_config.h.win32 claimed that Windows provides optreset.  Current
  mastodon results prove that in fact it does not; it was only because
  getopt.c defined the variable anyway that things failed to fall
  over.

- Remove a boatload of useless definitions of 'int optreset'.  If we
  are using our own ports of getopt or getopt_long, those will define
  the variable for themselves; and if not, we don't need these,
  because we never touch the variable anyway.

- Change EXPLAIN output so that subplans and initplans (particularly
  CTEs) are individually labeled, rather than just grouped under an
  "InitPlan" or "SubPlan" heading.  This in turn makes it possible for
  decompilation of a subplan reference to usefully identify which
  subplan it's referencing.  I also made InitPlans identify which
  parameter symbol(s) they compute, so that references to those
  parameters elsewhere in the plan tree can be connected to the
  initplan that will be executed.  Per a gripe from Robert Haas about
  EXPLAIN output of a WITH query being inadequate, plus some
  longstanding pet peeves of my own.

- In pgsql/src/backend/executor/execQual.c, make ExecInitExpr build
  the list of SubPlans found in a plan tree in order of discovery,
  rather than reverse order.  This doesn't matter functionally (I
  suppose the previous coding dates from the time when lcons was
  markedly cheaper than lappend).  However now that EXPLAIN is
  labeling subplans with IDs that are based on order of creation, this
  may help produce a slightly less surprising printout.

- Change cardinality() into a C-code function, instead of a
  SQL-language alias for array_length(v,1).  The efficiency gain here
  is doubtless negligible --- what I'm interested in is making sure
  that if we have second thoughts about the definition, we will not
  have to force a post-beta initdb to change the implementation.

Alvaro Herrera committed:

- In pgsql/doc/src/sgml/plpython.sgml, update URL to Python bug
  tracker.  Backpatch to 8.3; doesn't seem worthy of further
  backpatch.

- Disallow setting fillfactor for TOAST tables.  To implement this
  without almost duplicating the reloption table, treat relopt_kind as
  a bitmask instead of an integer value.  This decreases the range of
  allowed values, but it's not clear that there's need for that much
  values anyway.  This patch also makes heap_reloptions explicitly a
  no-op for relation kinds other than heap and TOAST tables.  Patch by
  ITAGAKI Takahiro with minor edits from me.  (In particular I removed
  the bit about adding relation kind to an error message, which I
  intend to commit separately.)

Bruce Momjian committed:

- In pgsql/doc/src/sgml/release.sgml, reorder release note sections.

- In pgsql/doc/src/sgml/release.sgml, more release note wording
  improvements;  section order adjustments.

- In pgsql/doc/src/sgml/release.sgml, more release note adjustments,
  reordering.

- In pgsql/doc/src/sgml/release.sgml, more release note changes,
  including a lower level of subsections.

- In pgsql/doc/src/sgml/release.sgml, more new subsections in release
  notes.

- In pgsql/doc/src/sgml/release.sgml, update release note introductory
  description.

- In pgsql/doc/src/sgml/release.sgml, remove some "Other" sections in
  the release notes by putting the items at the top of their sections.

- Change psql \d* display so 'S' _or_ a pattern include system
  objects.

- In pgsql/src/bin/psql/describe.c, do not show information_schema in
  \d* commands, unless 'S' or pattern is specified.  Martin Pihlak.

- Add support for additional DTrace probes.  Robert Lor.

- Have PL/pgSQL FETCH set DIAGNOSTICS ROW_COUNT.  Andrew (RhodiumToad)
  Gierth.

- In pgsql/src/backend/utils/misc/guc.c, give a better error message
  when trying to change "effective_io_concurrency" on systems without
  posix_fadvise().

- Revert DTrace patch from Robert Lor.

- In pgsql/doc/src/sgml/config.sgml, document that Solaris can't use
  effective_io_concurrency because of an ineffective posix_fadvise().

Heikki Linnakangas committed:

- In pgsql/src/backend/storage/ipc/procarray.c, fix a rare race
  condition when commit_siblings > 0 and a transaction commits at the
  same instant as a new backend is spawned.  Since CountActiveBackends()
  doesn't hold ProcArrayLock, it needs to be prepared for the case
  that a pointer at the end of the proc array is still NULL even
  though numProcs says it should be valid, since it doesn't hold
  ProcArrayLock.  Backpatch to 8.1.  8.0 and earlier had this right,
  but it was broken in the split of PGPROC and sinval shared memory
  arrays.  Per report and proposal by Marko Kreen.

- In pgsql/src/backend/utils/adt/pg_locale.c, update comment to
  reflect that LC_COLLATE and LC_CTYPE are now per-database settings.

Magnus Hagander committed:

- In pgsql/src/bin/initdb/initdb.c, don't crash initdb when we fail to
  get the current username.  Give an error message and exit instead,
  like we do elsewhere...  Per report from Wez Furlong and Robert
  Treat.

- In pgsql/src/port/path.c, make directory name comparisons on Win32
  case insensitive.  This method will not catch all different ways
  since the locale handling in NTFS doesn't provide an easy way to do
  that, but it will hopefully solve the most common cases causing
  startup problems when the backend is found in the system PATH.
  Attempts to fix bug #4694.

Teodor Sigaev committed:

- In pgsql/contrib/hstore/hstore_io.c, fix memory allocation for
  output of hstore type.  Per report from Zhang Maosen.

- Fix infinite loop while checking of partial match in pending list.
  Improve comments.  Now GIN-indexable operators should be strict.
  Per Tom Lane's questions/suggestions.

== Rejected Patches (for now) ==

Hitoshi Harada's doc patch which changed HeapTupleSatisfiesNow to
HeapTupleSatisfiesVisibility in doc/src/sgml/pgstattuple.sgml.

== Pending Patches ==

Abhijit Menon-Sen sent in a patch to implement has_sequence_privilege().

Andrew (RhodiumToad) Gierth sent in a patch to psql which adds a
mention of \ef to the \? command's output.

K. Srinath sent in a patch which allows an index to cover all tables
in an inheritance hierarchy.

Pavel Stehule and Tom Lane sent in a couple of versions of a patch to
allow raising an exception from _PG_init per previous discussion.

Heikki Linnakangas sent in a patch to help with message encoding.

Pavel Stehule sent in a patch to transform array inputs in variadic
functions into standard parameters.

Teodor Sigaev sent in two patches intended to fix a bug in GiST
reported by Andrew (RhodiumToad) Gierth.

Fujii Masao sent in another revision of his pg_standby trigger patch.

Robert Haas sent in a patch to add SET NDISTINCT to ALTER COLUMN.

Bruce Momjian sent in another revision of his patch to handle
platforms which don't support effective_io_concurrency.

Martin Pihlak sent in a patch which adds a U (user) modifier to psql's
\d commands in order to be able to match patterns only on non-system
objects.

Alvaro Herrera sent in a patch to fix a reloptions bug reported by
Khee Chin.

Teodor Sigaev sent in a patch to fix a crash in GiST insertion of
pathological box data.

Zdenek Kotala sent in another patch to fix the misbehavior of
Solaris's getopt.