== PostgreSQL Weekly News - December 01 2013 ==
== PostgreSQL Product News ==
pg_monz, a Zabbix template for monitoring PostgreSQL, released.
http://pg-monz.github.io/pg_monz/index-en.html
Pyrseas 0.7.0, a toolkit for PostgreSQL version control, released.
https://github.com/pyrseas/Pyrseas
== PostgreSQL Jobs for December ==
http://archives.postgresql.org/pgsql-jobs/2013-12/threads.php
== PostgreSQL Local ==
FOSDEM PGDay, a one day conference held before FOSDEM in Brussels,
Belgium, will be on Jan 31st, 2014. Details:
http://fosdem2014.pgconf.eu/
CfP is open:
http://fosdem2014.pgconf.eu/callforpapers/
== PostgreSQL in the News ==
Planet PostgreSQL: http://planet.postgresql.org/
PostgreSQL Weekly News is brought to you this week by David Fetter
Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david@fetter.org, German language
to pwn@pgug.de, Italian language to pwn@itpug.org. Spanish language
to pwn@arpug.com.ar.
== Applied Patches ==
Michael Meskes pushed:
- Fix ecpg parsing of sizeof(). The last fix used the wrong
non-terminal to define valid types.
http://git.postgresql.org/pg/commitdiff/ef8b3b00b55ef1feedb352a8a26f94845b6437dc
- More improvement to comment parsing in ecpg. ECPG is not supposed
to allow and output nested comments in C. These comments are only
allowed in the SQL parts and must not be written into the C file.
Also the different handling of different comments is documented.
http://git.postgresql.org/pg/commitdiff/05b476c2983a8d38bf6a0254553b4a0776896aa7
- ECPG: Make the preprocessor emit ';' if the variable type for a list
of variables is varchar. This fixes this test case: int main(void) {
exec sql begin declare section; varchar a[50], b[50]; exec sql end
declare section; return 0; } Since varchars are internally turned
into custom structs and the type name is emitted for these variable
declarations, the preprocessed code previously had: struct varchar_1
{ ... } a _,_ struct varchar_2 { ... } b ; The comma in the
generated C file was a syntax error. There are no regression test
changes since it's not exercised. Patch by Boszormenyi Zoltan
<zb@cybertec.at>
http://git.postgresql.org/pg/commitdiff/b46fa321003e4d07d881c2583eb1126e8fadabdc
- ECPG: Simplify free_variable(). Patch by Boszormenyi Zoltan
<zb@cybertec.at>
http://git.postgresql.org/pg/commitdiff/f641fc86fbe72a84a9d0a32279c4f9941bde6b6a
- ECPG: Add EXEC SQL CLOSE C to the tests. Patch by Boszormenyi
Zoltan <zb@cybertec.at>
http://git.postgresql.org/pg/commitdiff/1ec4c56e76714d3894cfc301212ec98674720e3b
- ECPG: Free the malloc()'ed variables in the test so it comes out
clean on Valgrind runs. Patch by Boszormenyi Zoltan
<zb@cybertec.at>
http://git.postgresql.org/pg/commitdiff/db58e8ff7cec9a14418d6dccd33bbe186d10c1f4
- ECPG: Fix offset to NULL/size indicator array. Patch by Boszormenyi
Zoltan <zb@cybertec.at>
http://git.postgresql.org/pg/commitdiff/d2542f9270e9587f54a63fd6ffbdd124abc6f80c
- Documentation fix for ecpg. The latest fixes removed a limitation
that was still in the docs, so Zoltan updated the docs, too.
http://git.postgresql.org/pg/commitdiff/2390f2b2d3ef708c282846457b6fd837a0c08061
- ECPG: Fix searching for quoted cursor names case-sensitively. Patch
by Böszörményi Zoltán <zb@cybertec.at>
http://git.postgresql.org/pg/commitdiff/51867a0f9b5c6d15c541e19c202520e95c5950a0
Bruce Momjian pushed:
- Change SET LOCAL/CONSTRAINTS/TRANSACTION and ABORT behavior. Change
SET LOCAL/CONSTRAINTS/TRANSACTION behavior outside of a transaction
block from error (post-9.3) to warning. (Was nothing in <= 9.3.)
Also change ABORT outside of a transaction block from notice to
warning.
http://git.postgresql.org/pg/commitdiff/a6542a4b6870a019cd952d055d2e7af2da2fe102
- pg_buffercache docs: adjust order of fields. Adjust order of fields
to match view order. Jaime Casanova
http://git.postgresql.org/pg/commitdiff/9ef780d4d7b4466c8ba9ab5cdcd7d326a35ecbe1
- pg_upgrade: Handle default_transaction_read_only settings. Setting
default_transaction_read_only=true could prevent pg_upgrade from
completing, so prepend default_transaction_read_only=false to
PGOPTIONS.
http://git.postgresql.org/pg/commitdiff/e7d56aee2d1805cd9d280c0e6f8c8af80a13aa3e
Jeff Davis pushed:
- Add missing entry for session_preload_libraries in sample config.
The omission was apparently an oversight in the original patch.
http://git.postgresql.org/pg/commitdiff/7cc0ba9f17fdae610187411439e8c01b26da15ac
Heikki Linnakangas pushed:
- Handle domains over arrays like plain arrays in PL/python. Domains
over arrays are now converted to/from python lists when passed as
arguments or return values. Like regular arrays. This has some
potential to break applications that rely on the old behavior that
they are passed as strings, but in practice there probably aren't
many such applications out there. Rodolfo Campero
http://git.postgresql.org/pg/commitdiff/37364c63115a52b4dc7ea280aa5b358abd4a9c38
- Oops, forgot to "git add" last minute changes to regression test.
http://git.postgresql.org/pg/commitdiff/4c83e0353f2c396e9e89fee564d9a5b868d5c3b2
- Don't update relfrozenxid if any pages were skipped. Vacuum
recognizes that it can update relfrozenxid by checking whether it
has processed all pages of a relation. Unfortunately it performed
that check after truncating the dead pages at the end of the
relation, and used the new number of pages to decide whether all
pages have been scanned. If the new number of pages happened to be
smaller or equal to the number of pages scanned, it incorrectly
decided that all pages were scanned. This can lead to relfrozenxid
being updated, even though some pages were skipped that still
contain old XIDs. That can lead to data loss due to xid wraparounds
with some rows suddenly missing. This likely has escaped notice so
far because it takes a large number (~2^31) of xids being used to
see the effect, while a full-table vacuum before that would fix the
issue. The incorrect logic was introduced by commit
b4b6923e03f4d29636a94f6f4cc2f5cf6298b8c8. Backpatch this fix down to
8.4, like that commit. Andres Freund, with some modifications by
me.
http://git.postgresql.org/pg/commitdiff/82b43f7df2036d06b4410721f77512969846b6d0
- Fix plpython3 expected output. I neglected this in the previous
commit that updated the plpython2 output, which I forgot to "git
add" earlier. As pointed out by Rodolfo Campero and Marko Kreen.
http://git.postgresql.org/pg/commitdiff/4118f7e8ede8a3616189b53983aea293fd0b3cb1
- More GIN refactoring. Separate the insertion payload from the more
static portions of GinBtree. GinBtree now only contains information
related to searching the tree, and the information of what to insert
is passed separately. Add root block number to GinBtree, instead of
passing it around all the functions as argument. Split off
ginFinishSplit() from ginInsertValue(). ginFinishSplit is
responsible for finding the parent and inserting the downlink to it.
http://git.postgresql.org/pg/commitdiff/ce5326eed386959aac7a322880896ddeade7fd52
- Get rid of the post-recovery cleanup step of GIN page splits.
Replace it with an approach similar to what GiST uses: when a page
is split, the left sibling is marked with a flag indicating that the
parent hasn't been updated yet. When the parent is updated, the flag
is cleared. If an insertion steps on a page with the flag set, it
will finish split before proceeding with the insertion. The
post-recovery cleanup mechanism was never totally reliable, as
insertion to the parent could fail e.g because of running out of
memory or disk space, leaving the tree in an inconsistent state.
This also divides the responsibility of WAL-logging more clearly
between the generic ginbtree.c code, and the parts specific to entry
and posting trees. There is now a common WAL record format for
insertions and deletions, which is written by ginbtree.c, followed
by tree-specific payload, which is returned by the placetopage- and
split- callbacks.
http://git.postgresql.org/pg/commitdiff/631118fe1e8f66fd15eb0013dd9940d9146a5096
- Fix gin_desc routine to match the WAL format. In the GIN
incomplete-splits patch, I used BlockIdDatas to store the block
number of left and right children, when inserting a downlink after a
split to an internal page posting list page. But gin_desc thought
they were stored as BlockNumbers.
http://git.postgresql.org/pg/commitdiff/97c19e6c38ebc5e300539f6985ef3e05398c8f46
- Another gin_desc fix. The number of items inserted was incorrectly
printed as if it was a boolean.
http://git.postgresql.org/pg/commitdiff/2fe69cacff4e92201cb3e3de7748f3e1d51d9e25
Peter Eisentraut pushed:
- doc: Add id to index in XSLT build. That way, the HTML file name of
the index will be the same as currently for the DSSSL build.
http://git.postgresql.org/pg/commitdiff/3803ff985c7940af32912b158322cce399952e76
- Implement information_schema.parameters.parameter_default column.
Reviewed-by: Ali Dar <ali.munir.dar@gmail.com> Reviewed-by: Amit
Khandekar <amit.khandekar@enterprisedb.com> Reviewed-by: Rodolfo
Campero <rodolfo.campero@anachronics.com>
http://git.postgresql.org/pg/commitdiff/85ed91ee7da8f560c3443733cbda2862cd25facd
- doc: Put data types in alphabetical order. From: Andreas Karlsson
<andreas@proxel.se>
http://git.postgresql.org/pg/commitdiff/a607b6903e90383e408d5085ca9b93a8a22392c1
- doc: Set chunk.first.sections in XSLT, for consistency with DSSSL
output
http://git.postgresql.org/pg/commitdiff/b6ecde8ed4799720eaa1c015e325012dbad095fa
- doc: Allow selecting web site CSS style sheet in XSLT HTML build
http://git.postgresql.org/pg/commitdiff/384eb1d40db6fe97678cb00e57365e785cb85eff
- doc: Enhance documentation of ssl_ciphers setting a bit
http://git.postgresql.org/pg/commitdiff/50107ee7a58f36568997db84df641fde122f3132
- doc: Simplify handling of variablelists in XSLT build. The
previously used custom template is no longer necessary because
parameters provided by the standard style sheet can achieve the same
outcome.
http://git.postgresql.org/pg/commitdiff/1eafea5d1bc5cf934eaf96e65115841fe3829986
- Remove use of obsolescent Autoconf macros. Remove the use of the
following macros, which are obsolescent according to the Autoconf
documentation: AC_C_CONST AC_C_STRINGIZE AC_C_VOLATILE
AC_FUNC_MEMCMP
http://git.postgresql.org/pg/commitdiff/34fa72ec9cfc1f5c4b11b67b2ce7eac6caca9fa5
- doc: Disable preface.autolabel in XSLT. The makes the output more
consistent with the existing DSSSL setup.
http://git.postgresql.org/pg/commitdiff/3c81b5c1d28faaacbd56583ebf00dd5f3db57c31
- Translation updates
http://git.postgresql.org/pg/commitdiff/3e3520cf7a70aa85f55a7be37924aaa1809a4be3
Fujii Masao pushed:
- Fix typo in release note. Backpatch to 9.1. Josh Kupershmidt
http://git.postgresql.org/pg/commitdiff/551c78281b47b8ab12cfc142b8da68db8473e591
- Add --xlogdir option to pg_basebackup, for specifying the pg_xlog
directory. Haribabu kommi, slightly modified by me.
http://git.postgresql.org/pg/commitdiff/d1b88f6b3660753771895a4441b6bb4ccc9bcaac
Tom Lane pushed:
- Minor corrections in lmgr/README. Correct an obsolete statement
that no backend touches another backend's PROCLOCK lists. This was
probably wrong even when written (the deadlock checker looks at
everybody's lists), and it's certainly quite wrong now that
fast-path locking can require creation of lock and proclock objects
on behalf of another backend. Also improve some statements in the
hot standby explanation, and do one or two other trivial bits of
wordsmithing/ reformatting.
http://git.postgresql.org/pg/commitdiff/8c84803e14b1ba2025c37dcc1ff1c41dd6264fa0
- Fix stale-pointer problem in fast-path locking logic. When
acquiring a lock in fast-path mode, we must reset the locallock
object's lock and proclock fields to NULL. They are not necessarily
that way to start with, because the locallock could be left over
from a failed lock acquisition attempt earlier in the transaction.
Failure to do this led to all sorts of interesting misbehaviors when
LockRelease tried to clean up no-longer-related lock and proclock
objects in shared memory. Per report from Dan Wood. In passing,
modify LockRelease to elog not just Assert if it doesn't find lock
and proclock objects for a formerly fast-path lock, matching the
code in FastPathGetRelationLockEntry and LockRefindAndRelease. This
isn't a bug but it will help in diagnosing any future bugs in this
area. Also, modify FastPathTransferRelationLocks and
FastPathGetRelationLockEntry to break out of their loops over the
fastpath array once they've found the sole matching entry. This was
inconsistently done in some search loops and not others. Improve
assorted related comments, too. Back-patch to 9.2 where the
fast-path mechanism was introduced.
http://git.postgresql.org/pg/commitdiff/7db285afc9cf51763c5f49c453f8d189dca17b9d
- Fix latent(?) race condition in LockReleaseAll. We have for a long
time checked the head pointer of each of the backend's proclock
lists and skipped acquiring the corresponding locktable partition
lock if the head pointer was NULL. This was safe enough in the days
when proclock lists were changed only by the owning backend, but it
is pretty questionable now that the fast-path patch added cases
where backends add entries to other backends' proclock lists.
However, we don't really wish to revert to locking each partition
lock every time, because in simple transactions that would add a lot
of useless lock/unlock cycles on already-heavily-contended LWLocks.
Fortunately, the only way that another backend could be modifying
our proclock list at this point would be if it was promoting a
formerly fast-path lock of ours; and any such lock must be one that
we'd decided not to delete in the previous loop over the locallock
table. So it's okay if we miss seeing it in this loop; we'd just
decide not to delete it again. However, once we've detected a
non-empty list, we'd better re-fetch the list head pointer after
acquiring the partition lock. This guards against possibly fetching
a corrupt-but-non-null pointer if pointer fetch/store isn't atomic.
It's not clear if any practical architectures are like that, but
we've never assumed that before and don't wish to start here. In
any case, the situation certainly deserves a code comment. While at
it, refactor the partition traversal loop to use a for() construct
instead of a while() loop with goto's. Back-patch, just in case the
risk is real and not hypothetical.
http://git.postgresql.org/pg/commitdiff/da8a7160893c463690146c7b30220b90c593ec15
- Fix assorted race conditions in the new timeout infrastructure.
Prevent handle_sig_alarm from losing control partway through due to
a query cancel (either an asynchronous SIGINT, or a cancel triggered
by one of the timeout handler functions). That would at least
result in failure to schedule any required future interrupt, and
might result in actual corruption of timeout.c's data structures, if
the interrupt happened while we were updating those. We could still
lose control if an asynchronous SIGINT arrives just as the function
is entered. This wouldn't break any data structures, but it would
have the same effect as if the SIGALRM interrupt had been silently
lost: we'd not fire any currently-due handlers, nor schedule any new
interrupt. To forestall that scenario, forcibly reschedule any
pending timer interrupt during AbortTransaction and
AbortSubTransaction. We can avoid any extra kernel call in most
cases by not doing that until we've allowed LockErrorCleanup to kill
the DEADLOCK_TIMEOUT and LOCK_TIMEOUT events. Another hazard is
that some platforms (at least Linux and *BSD) block a signal before
calling its handler and then unblock it on return. When we longjmp
out of the handler, the unblock doesn't happen, and the signal is
left blocked indefinitely. Again, we can fix that by forcibly
unblocking signals during AbortTransaction and AbortSubTransaction.
These latter two problems do not manifest when the longjmp reaches
postgres.c, because the error recovery code there kills all pending
timeout events anyway, and it uses sigsetjmp(..., 1) so that the
appropriate signal mask is restored. So errors thrown outside any
transaction should be OK already, and cleaning up in
AbortTransaction and AbortSubTransaction should be enough to fix
these issues. (We're assuming that any code that catches a query
cancel error and doesn't re-throw it will do at least a
subtransaction abort to clean up; but that was pretty much required
already by other subsystems.) Lastly, ProcSleep should not clear the
LOCK_TIMEOUT indicator flag when disabling that event: if a lock
timeout interrupt happened after the lock was granted, the ensuing
query cancel is still going to happen at the next
CHECK_FOR_INTERRUPTS, and we want to report it as a lock timeout not
a user cancel. Per reports from Dan Wood. Back-patch to 9.3 where
the new timeout handling infrastructure was introduced. We may at
some point decide to back-patch the signal unblocking changes
further, but I'll desist from that until we hear actual field
complaints about it.
http://git.postgresql.org/pg/commitdiff/16e1b7a1b7f7ffd8a18713e83c8cd72c9ce48e07
- Be sure to release proc->backendLock after SetupLockInTable()
failure. The various places that transferred fast-path locks to the
main lock table neglected to release the PGPROC's backendLock if
SetupLockInTable failed due to being out of shared memory. In most
cases this is no big deal since ensuing error cleanup would release
all held LWLocks anyway. But there are some hot-standby functions
that don't consider failure of FastPathTransferRelationLocks to be a
hard error, and in those cases this oversight could lead to system
lockup. For consistency, make all of these places look the same as
FastPathTransferRelationLocks. Noted while looking for the cause of
Dan Wood's bugs --- this wasn't it, but it's a bug anyway.
http://git.postgresql.org/pg/commitdiff/8b151558c837ea5fccccd84adde251eeddde81f0
- Fix assorted issues in pg_ctl's pgwin32_CommandLine(). Ensure that
the invocation command for postgres or pg_ctl runservice
double-quotes the executable's pathname; failure to do this leads to
trouble when the path contains spaces. Also, ensure that the path
ends in ".exe" in both cases and uses backslashes rather than
slashes as directory separators. The latter issue is reported to
confuse some third-party tools such as Symantec Backup Exec. Also,
rewrite the function to avoid buffer overrun issues by using a
PQExpBuffer instead of a fixed-size static buffer. Combinations of
very long executable pathnames and very long data directory
pathnames could have caused trouble before, for example. Back-patch
to all active branches, since this code has been like this for a
long while. Naoya Anzai and Tom Lane, reviewed by Rajeev Rastogi
http://git.postgresql.org/pg/commitdiff/79193c75f8c78ff494f32cd7b7ee79d7166c225b
- Editorial corrections to the October 2013 minor-release notes. This
is mostly to fix incorrect migration instructions: since the
preceding minor releases advised reindexing some GIST indexes, it's
important that we back-link to that advice rather than earlier
instances. Also improve some bug descriptions and fix a few typos.
No back-patch yet; these files will get copied into the back
branches later in the release process.
http://git.postgresql.org/pg/commitdiff/47960354024eac823981d7f264a0cf804b0c42c9
- Update time zone data files to tzdata release 2013h. DST law
changes in Argentina, Brazil, Jordan, Libya, Liechtenstein, Morocco,
Palestine. New timezone abbreviations WIB, WIT, WITA for Indonesia.
http://git.postgresql.org/pg/commitdiff/335470251d69c5c63a864c706db02fe173576834
- Draft release notes for 9.3.2. I'm putting these up for review
before I start to extract the relevant subsets for the older
branches. It'll be easier to make any suggested wording
improvements at this stage.
http://git.postgresql.org/pg/commitdiff/23e796de15567e9d31e8f9e8661828179f24a7be
Kevin Grittner pushed:
- Minor correction of READ COMMITTED isolation level docs. Per report
from AK.
http://git.postgresql.org/pg/commitdiff/89ba815092f9ab0b153aa67328a7c18431d809d7
- Fix pg_dumpall to work for databases flagged as read-only.
pg_dumpall's charter is to be able to recreate a database cluster's
contents in a virgin installation, but it was failing to honor that
contract if the cluster had any ALTER DATABASE SET
default_transaction_read_only settings. By including a SET command
for the connection for each connection opened by pg_dumpall output,
errors are avoided and the source cluster is successfully recreated.
There was discussion of whether to also set this for the connection
applying pg_dump output, but it was felt that it was both less
appropriate in that context, and far easier to work around.
Backpatch to all supported branches.
http://git.postgresql.org/pg/commitdiff/4bd371f6f886446b6c0886f088992e6f3d4ab4a2
Alvaro Herrera pushed:
- Cope with heap_fetch failure while locking an update chain. The
reason for the fetch failure is that the tuple was removed because
it was dead; so the failure is innocuous and can be ignored.
Moreover, there's no need for further work and we can return success
to the caller immediately. EvalPlanQualFetch is doing something
very similar to this already. Report and test case from Andres
Freund in 20131124000203.GA4403@alap2.anarazel.de
http://git.postgresql.org/pg/commitdiff/e518fa7adfc28e0d0a99fe4e5711386d9a5c6532
- Compare Xmin to previous Xmax when locking an update chain. Not
doing so causes us to traverse an update chain that has been broken
by concurrent page pruning. All other code that traverses update
chains uses this check as one of the cases in which to stop
iterating, so replicate it here too. Failure to do so leads to
erroneous CLOG, subtrans or multixact lookups. Per discussion
following the bug report by J Smith in
CADFUPgc5bmtv-yg9znxV-vcfkb+JPRqs7m2OesQXaM_4Z1JpdQ@mail.gmail.com
as diagnosed by Andres Freund.
http://git.postgresql.org/pg/commitdiff/e4828e9ccba731178dd77aed078db7ceb0e1e8d1
- Use a more granular approach to follow update chains. Instead of
simply checking the KEYS_UPDATED bit, we need to check whether each
lock held on the future version of the tuple conflicts with the lock
we're trying to acquire. Per bug report #8434 by Tomonari Katsumata
http://git.postgresql.org/pg/commitdiff/247c76a989097f1b4ab6fae898f24e75aa27fc1b
- Don't try to set InvalidXid as page pruning hint. If a transaction
updates/deletes a tuple just before aborting, and a concurrent
transaction tries to prune the page concurrently, the pruner may see
HeapTupleSatisfiesVacuum return HEAPTUPLE_DELETE_IN_PROGRESS, but a
later call to HeapTupleGetUpdateXid() return InvalidXid. This would
cause an assertion failure in development builds, but would be
otherwise Mostly Harmless. Fix by checking whether the updater Xid
is valid before trying to apply it as page prune point. Reported by
Andres in 20131124000203.GA4403@alap2.anarazel.de
http://git.postgresql.org/pg/commitdiff/c235a6a589bbd0a24e54fdb0df28979c9fb09463
- Unbreak buildfarm. I removed an intermediate commit before pushing
and forgot to test the resulting tree :-(
http://git.postgresql.org/pg/commitdiff/d51a8c52bae6f95af100572a1df94cc3362d2059
- Don't TransactionIdDidAbort in HeapTupleGetUpdateXid. It is
dangerous to do so, because some code expects to be able to see
what's the true Xmax even if it is aborted (particularly while
traversing HOT chains). So don't do it, and instead rely on the
callers to verify for abortedness, if necessary. Several race
conditions and bugs fixed in the process. One isolation test
changes the expected output due to these. This also reverts commit
c235a6a589b, which is no longer necessary. Backpatch to 9.3, where
this function was introduced. Andres Freund
http://git.postgresql.org/pg/commitdiff/1ce150b7bb14105ddc190c5f1acf2ae1a9b2854a
- Truncate pg_multixact/'s contents during crash recovery. Commit
9dc842f08 of 8.2 era prevented MultiXact truncation during crash
recovery, because there was no guarantee that enough state had been
setup, and because it wasn't deemed to be a good idea to remove data
during crash recovery anyway. Since then, due to Hot-Standby,
streaming replication and PITR, the amount of time a cluster can
spend doing crash recovery has increased significantly, to the point
that a cluster may even never come out of it. This has made not
truncating the content of pg_multixact/ not defensible anymore. To
fix, take care to setup enough state for multixact truncation before
crash recovery starts (easy since checkpoints contain the required
information), and move the current end-of-recovery actions to a new
TrimMultiXact() function, analogous to TrimCLOG(). At some later
point, this should probably done similarly to the way clog.c is
doing it, which is to just WAL log truncations, but we can't do that
for the back branches. Back-patch to 9.0. 8.4 also has the
problem, but since there's no hot standby there, it's much less
pressing. In 9.2 and earlier, this patch is simpler than in newer
branches, because multixact access during recovery isn't required.
Add appropriate checks to make sure that's not happening. Andres
Freund
http://git.postgresql.org/pg/commitdiff/1df0122daa6510eed4146033379a5055f66f5a8e
- Fix full-table-vacuum request mechanism for MultiXactIds. While
autovacuum dutifully launched anti-multixact-wraparound vacuums when
the multixact "age" was reached, the vacuum code was not aware that
it needed to make them be full table vacuums. As the resulting
partial-table vacuums aren't capable of actually increasing
relminmxid, autovacuum continued to launch anti-wraparound vacuums
that didn't have the intended effect, until age of relfrozenxid
caused the vacuum to finally be a full table one via
vacuum_freeze_table_age. To fix, introduce logic for multixacts
similar to that for plain TransactionIds, using the same GUCs.
Backpatch to 9.3, where permanent MultiXactIds were introduced.
Andres Freund, some cleanup by Álvaro
http://git.postgresql.org/pg/commitdiff/f54106f77e6d71cbb3fa0924095e5142341fde2b
- Replace hardcoded 200000000 with autovacuum_freeze_max_age. Parts
of the code used autovacuum_freeze_max_age to determine whether
anti-multixact-wraparound vacuums are necessary, while others used a
hardcoded 200000000 value. This leads to problems when
autovacuum_freeze_max_age is set to a non-default value. Use the
latter everywhere. Backpatch to 9.3, where vacuuming of multixacts
was introduced. Andres Freund
http://git.postgresql.org/pg/commitdiff/76a31c689c627268067175b5d0687ce0dac9a4f4
- Fix a couple of bugs in MultiXactId freezing. Both
heap_freeze_tuple() and heap_tuple_needs_freeze() neglected to look
into a multixact to check the members against cutoff_xid. This
means that a very old Xid could survive hidden within a multi,
possibly outliving its CLOG storage. In the distant future, this
would cause clog lookup failures: ERROR: could not access status of
transaction 3883960912 DETAIL: Could not open file "pg_clog/0E78":
No such file or directory. This mostly was problematic when the
updating transaction aborted, since in that case the row wouldn't
get pruned away earlier in vacuum and the multixact could possibly
survive for a long time. In many cases, data that is inaccessible
for this reason way can be brought back heuristically. As a second
bug, heap_freeze_tuple() didn't properly handle multixacts that need
to be frozen according to cutoff_multi, but whose updater xid is
still alive. Instead of preserving the update Xid, it just set Xmax
invalid, which leads to both old and new tuple versions becoming
visible. This is pretty rare in practice, but a real threat
nonetheless. Existing corrupted rows, unfortunately, cannot be
repaired in an automated fashion. Existing physical replicas might
have already incorrectly frozen tuples because of different behavior
than in master, which might only become apparent in the future once
pg_multixact/ is truncated; it is recommended that all clones be
rebuilt after upgrading. Following code analysis caused by bug
report by J Smith in message
CADFUPgc5bmtv-yg9znxV-vcfkb+JPRqs7m2OesQXaM_4Z1JpdQ@mail.gmail.com
and privately by F-Secure. Backpatch to 9.3, where freezing of
MultiXactIds was introduced. Analysis and patch by Andres Freund,
with some tweaks by Álvaro.
http://git.postgresql.org/pg/commitdiff/2393c7d102368717283d7121a6ea8164e902b011
Robert Haas pushed:
- Refine our definition of what constitutes a system relation.
Although user-defined relations can't be directly created in
pg_catalog, it's possible for them to end up there, because you can
create them in some other schema and then use ALTER TABLE .. SET
SCHEMA to move them there. Previously, such relations couldn't
afterwards be manipulated, because
IsSystemRelation()/IsSystemClass() rejected all attempts to modify
objects in the pg_catalog schema, regardless of their origin. With
this patch, they now reject only those objects in pg_catalog which
were created at initdb-time, allowing most operations on
user-created tables in pg_catalog to proceed normally. This patch
also adds new functions IsCatalogRelation() and IsCatalogClass(),
which is similar to IsSystemRelation() and IsSystemClass() but with
a slightly narrower definition: only TOAST tables of system catalogs
are included, rather than *all* TOAST tables. This is currently
used only for making decisions about when invalidation messages need
to be sent, but upcoming logical decoding patches will find other
uses for this information. Andres Freund, with some modifications
by me.
http://git.postgresql.org/pg/commitdiff/8e18d04d4daf34b8a557e2dc553a7754b255cd9a
== Rejected Patches (for now) ==
No one was disappointed this week :-)
== Pending Patches ==
Heikki Linnakangas sent in a patch to critique an optimization for
fixed-length arrays in PL/pgsql.
Kyotaro HORIGUCHI sent in new revisions of two ways to optimize
certain UNION ALL queries.
Rajeev Rastogi sent in three more revisions of a patch to ensure that
COPY FROM ... STDIN generates a count tag.
Stephen Frost and Dimitri Fontaine traded patches to implement
EXTENSION templates.
SAKAMOTO Masahiko sent in three more revisions of a patch to enable
logging WAL when updating hint bits.
Peter Eisentraut sent in another revision of a patch to add EVENT
triggers to PL/Perl.
Gurjeet Singh sent in a patch to remove "Nothing to be done for 'all'"
messages.
Heikki Linnakangas sent in another revision of a patch to implement
INSERT...ON DUPLICATE KEY LOCK FOR UPDATE.
Kyotaro HORIGUCHI and Etsuro Fujita traded patches to improve the path
generation for certain indexed queries.
Rajeev Rastogi and Amit Kapila traded revisions of a patch to split
out pg_resetxlog output into pre- and post-sections.
Haribabu Kommi sent in two more revisions of a patch to ensure that a
heavily modified table not bloat in the case where autovacuum is
having a hard time keeping up.
Pavel Stehule sent in another revision of a patch to detect orphan
locks.
Gurjeet Singh sent in another revision of a patch to shave a few
instructions from backend child startup.
Marko Kreen sent in a patch to fix the regression tests for PL/Python3
per the addition of domains over arrays.
Tom Dunstan sent in a patch to allow foreign keys to have WHERE
clauses.
Bruce Momjian sent in a patch to fix an issue in PostgreSQL 9.2 where
during index-only scans, there were abnormal heap fetches after VACUUM
FULL.
Pavel Stehule sent in another revision of a patch to implement new
border styles for psql.
Pavel Stehule and Dean Rasheed traded patches to expand the DROP ...
IF EXISTS construct to more database objects.
Alvaro Herrera sent in a patch to avoid a useless palloc().
KaiGai Kohei sent in another flock of patches implementing and using
custom plan nodes.
Robert Haas sent in another revision of a patch to implement logical
changeset logging.
Gregory Stark sent in another revision of a WIP patch to optimize away
some overflow checks.
Heikki Linnakangas sent in two more revisions of a patch to implement
GIN packed posting lists.
Fabrízio de Royes Mello sent in another revision of a patch to
implement time-delayed standbys.
Bruce Momjian sent in a patch to fix an issue where pg_upgrades
segfaults when given an invalid PGSERVICE value.
Bruce Momjian sent in a patch to document some behavior around client
certificates.
Peter Geoghegan sent in two more revisions of a patch to add some
features to pg_stat_statements.