== PostgreSQL Weekly News - December 12 2010 ==

== PostgreSQL Weekly News - December 12 2010 ==

The Call for Papers is still open for the PostgreSQL room at FOSDEM,
which will be held in Brussels, Belgium, on February 5-6, 2011.  Send
your proposals to fosdem AT postgresql DOT eu.

Devrim GUNDUZ sent in a fix for the RHEL6 RPMs.  See below for
details.
http://people.planetpostgresql.org/devrim/index.php?/archives/54-Attention-9.0.1-OR-RHEL-6-RPM-users.html

== PostgreSQL Product News ==

PostgreSQL Maestro 10.12, a GUI admin tool for PostgreSQL, released.
http://www.sqlmaestro.com/products/postgresql/maestro/

psycopg2 2.3.1, a Python connector for PostgreSQL, released.
http://initd.org/psycopg/articles/2010/12/04/psycopg-231-released/

Slony-I, a master-to-multiple-cascading-replica system for PostgreSQL,
released updated revisions 1.2.22 and 2.0.6.
http://www.slony.info/

== PostgreSQL Jobs for December ==

http://archives.postgresql.org/pgsql-jobs/2010-12/threads.php

== PostgreSQL Local ==

The PostgreSQLFr Call for Projects has begun.  Projects must target
PostgreSQL and the French-speaking community.  Mail appel-projets-2010
AT postgresql DOT fr.
http://www.postgresql.fr/appel_a_projets_2010:call_for_projects

FOSDEM is one of the biggest Free and Open Source event held annually
in Brussels, Belgium, on February 5-6, 2011, and attended by around
4000 people.  Send in your proposal to fosdem AT postgresql DOT eu.
http://andreas.scherbaum.la/blog/archives/749-PostgreSQLFOSDEM-2011-Call-for-talks.html

PGDay is scheduled at this year's Southern California Linux Exposition
(SCALE) held in the LAX Hilton Hotel in the city of Los Angeles,
California, on Friday February 25th, 2011.  Post your talk submission
to pgday-submissions AT googlegroups DOT com.

PGCon will be May 19-20, 2011 at the University of Ottawa, preceded by
two days of tutorials on May 17-18.
http://www.pgcon.org/2011/


== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david@fetter.org, German language
to pwn@pgug.de, Italian language to pwn@itpug.org.  Spanish language
to pwn@arpug.com.ar.

== Reviews ==

ITAGAKI Takahiro reviewed the per-column collation patch, bringing up
issues with pg_dump, along with a couple of naming issues in the
source code and a request for clearer global state.
== Applied Patches ==

Peter Eisentraut pushed:

- Put only single space after "Sort Method:", for consistency.
  http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=951d7861212120571a601d946194892db6f2bde1

Heikki Linnakangas pushed:

- Fix two typos, by Fujii Masao.
  http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=95e42a2c2980003880dee622fc3bed0b03322542

- Fix bugs in the hot standby known-assigned-xids tracking logic. If
  there's an old transaction running in the master, and a lot of
  transactions have started and finished since, and a WAL-record is
  written in the gap between the creating the running-xacts snapshot
  and WAL-logging it, recovery will fail with "too many
  KnownAssignedXids" error.  This bug was reported by Joachim Wieland
  on Nov 19th.  In the same scenario, when fewer transactions have
  started so that all the xids fit in KnownAssignedXids despite the
  first bug, a more serious bug arises.  We incorrectly initialize the
  clog code with the oldest still running transaction, and when we see
  the WAL record belonging to a transaction with an XID larger than
  one that committed already before the checkpoint we're recovering
  from, we zero the clog page containing the already committed
  transaction, leading to data loss.  In hindsight, trying to track
  xids in the known-assigned-xids array before seeing the
  running-xacts record was too complicated.  To fix that, hold
  XidGenLock while the running-xacts snapshot is taken and WAL-logged.
  That ensures that no transaction can begin or end in that gap, so
  that in recvoery we know that the snapshot contains all transactions
  running at that point in WAL.
  http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=5a031a5556ff83b8a9646892715d7fef415b83c3

Andrew Dunstan pushed:

- Allow the low level COPY routines to read arbitrary numbers of
  fields.  This doesn't involve any user-visible change in behavior,
  but will be useful when the COPY routines are exposed to allow their
  use by Foreign Data Wrapper routines, which will be able to use
  these routines to read irregular CSV files, for example.
  http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=af1a614ec6d074fdea46de2e1c462f23fc7ddc6f

Tom Lane pushed:

- Add a stack overflow check to copyObject().  There are some code
  paths, such as SPI_execute(), where we invoke copyObject() on raw
  parse trees before doing parse analysis on them.  Since the bison
  grammar is capable of building heavily nested parsetrees while
  itself using only minimal stack depth, this means that copyObject()
  can be the front-line function that hits stack overflow before
  anything else does.  Accordingly, it had better have a
  check_stack_depth() call.  I did a bit of performance testing and
  found that this slows down copyObject() by only a few percent, so
  the hit ought to be negligible in the context of complete processing
  of a query.  Per off-list report from Toshihide Katayama.
  Back-patch to all supported branches.
  http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=8b5692809707c0e15d04c530a3fed9347350ea01

- Force default wal_sync_method to be fdatasync on Linux.  Recent
  versions of the Linux system header files cause xlogdefs.h to
  believe that open_datasync should be the default sync method,
  whereas formerly fdatasync was the default on Linux.  open_datasync
  is a bad choice, first because it doesn't actually outperform
  fdatasync (in fact the reverse), and second because we try to use
  O_DIRECT with it, causing failures on certain filesystems (e.g.,
  ext4 with data=journal option).  This part of the patch is largely
  per a proposal from Marti Raudsepp.  More extensive changes are
  likely to follow in HEAD, but this is as much change as we want to
  back-patch.  Also clean up confusing code and incorrect
  documentation surrounding the fsync_writethrough option.  Those
  changes shouldn't result in any actual behavioral change, but I
  chose to back-patch them anyway to keep the branches looking similar
  in this area.  In 9.0 and HEAD, also do some copy-editing on the WAL
  Reliability documentation section.  Back-patch to all supported
  branches, since any of them might get used on modern Linux versions.
  http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=576477e73c4a9cb32c7471c1cec354fd5328140e

- Clean up some copied-and-pasted code in pg_upgrade.  1. Don't
  reimplement S_ISDIR() and S_ISREG() badly.  2. Don't reimplement
  access() badly.  This code appears to have been copied from ancient
  versions of the corresponding backend routines, and not patched to
  incorporate subsequent fixes (see my commits of 2008-03-31 and
  2010-01-14 respectively).  It might be a good idea to change it to
  just *call* those routines, but for now I'll just transpose these
  fixes over.
  http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=663fc32e26e8df41434d751e2203c1aa410d1916

- Make S_IRGRP etc available in mingw builds as well as MSVC.  (Hm, I
  wonder whether BCC defines them either...)  Also label dangling
  endifs a bit better in this area.
  http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=5132ad8bdf8e85ea0054217ef7ee38c51ace4824

- Reset all database-level stats in pgstat_recv_resetcounter().  We
  were failing to zero out some pg_stat_database counters that have
  been added since the initial pgstats coding.  This is a bug, but not
  back-patching the fix since changing this behavior in a minor
  release seems a cure worse than the disease.  Report and patch by
  Tomas Vondra.
  http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=0c90442355fbbe785740669f63141f24674c8958

Simon Riggs pushed:

- Optimize commit_siblings in two ways to improve group commit.
  First, avoid scanning the whole ProcArray once we know there are at
  least commit_siblings active; second, skip the check altogether if
  commit_siblings = 0.  Greg Smith.
  http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=e620ee35b249b0af255ef788003d1c9edb815a35

- Reduce spurious Hot Standby conflicts from never-visible records.
  Hot Standby conflicts only with tuples that were visible at some
  point.  So ignore tuples from aborted transactions or for tuples
  updated/deleted during the inserting transaction when generating the
  conflict transaction ids.  Following detailed analysis and test case
  by Noah Misch.  Original report covered btree delete records,
  correctly observed by Heikki Linnakangas that this applies to other
  cases also.  Fix covers all sources of cleanup records via common
  code.
  http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=b9075a6d2f9b07a00262a670dd60272904c79dce

- Self review of previous patch.  Fix assumption that xmax >= xmin.
  http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=9975c683b102d06ed5d5ab799eaba0d00a9ff38c

- Reduce spurious Hot Standby conflicts from never-visible records.
  Hot Standby conflicts only with tuples that were visible at some
  point. So ignore tuples from aborted transactions or for tuples
  updated/deleted during the inserting transaction when generating the
  conflict transaction ids.  Following detailed analysis and test case
  by Noah Misch.  Original report covered btree delete records,
  correctly observed by Heikki Linnakangas that this applies to other
  cases also.  Fix covers all sources of cleanup records via common
  code.  Includes additional fix compared to commit on HEAD
  http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=a804a23e7af0e075b88e7b03bfd9b0f22d2657b2

Robert Haas pushed:

- Minor documentation cleanup.  Fujii Masao
  http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=1490946c57f44d3bf8601e9a404aaa80b58a715a

- Allow bidirectional copy messages in streaming replication mode.
  Fujii Masao.  Review by Alvaro Herrera, Tom Lane, and myself.
  http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=d3d414696f39e2b57072fab3dd4fa11e465be4ed

Magnus Hagander pushed:

- Add required new port files to MSVC builds.
  http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=20f396429186a35a81fc7ef3ad34c3134ead2992

== Rejected Patches (for now) ==

No one was disappointed this week :-)

== Pending Patches ==

Joachim Wieland sent in another revision of the patch to clone
snapshots and make them available.

Craig Ringer sent in a patch to add a crash dump, similar to a core
dump, only for Windows.

Josh Berkus sent in two revisions of a patch to make wal_sync_method
default to fdatasync on Linux kernels 2.6.33 and higher.

Fujii Masao sent in another revision of the patch to change libpq to
help with synchronous replication.

Fujii Masao sent in a patch to set a replication timeout.  This, too,
is intended to help with synchronous replication.

Peter Eisentraut sent in a patch to add a pg_sequence_parameters()
function.

KaiGai Kohei sent in another revision of the access control label
switcher function.

Joachim Wieland sent in another revision of the patch to add a
directory-based format for pg_dump/pg_restore.

Fujii Masao sent in a patch to remove wal_sender_delay.

Tim Bunce sent in a patch to optimize argument-passing in PL/Perl[U].

Kevin Grittner sent in four revisions of a patch to add a optimize and
correct SERIALIZABLE so that it doesn't switch from READ ONLY
DEFERRABLE mode to READ WRITE.

Tom Lane sent in a patch to fix some infelicities in the speed of
calculating percent_rank() vs. rank().

Fujii Masao sent in a patch to fix the way ps(1) displays standby
queries.

Dimitri Fontaine sent in two more revisions to add extensions.

Robert Haas sent in another revision of the patch to add unlogged
tables.

Thomas (tv AT fuzzy DOT cz) sent in two revisions of a patch to keep a
timestamp of the most recent stats reset per-database, per-table, and
per-function.

Dimitri Fontaine sent in another revision of the patch to add a
pg_execute_from_file() function.  This is part of the EXTENSIONS work.

Alexander Korotkov sent in a patch to add wildcard support for
pg_trgm.

Simon Riggs sent in a patch to add a NOT ENFORCED clause to foreign
key constraints.

Peter Eisentraut sent in another revision of the per-column collation
patch.

ITAGAKI Takahiro sent in another revision of the patch to add
file_fdw(), part of the SQL/MED standard dealing with reading and
writing external files.