Обсуждение: Assertion failure on HEAD (or at least git copy of it)

Checking out HEAD from the git repository I see an assertion
failure.  Steps to reproduce:
make distclean
./configure --prefix=/usr/local/pgsql-serializable \ --enable-debug --enable-depend --enable-cassert
make check
sudo make install
cd contrib/
make
sudo make install
cd ../
rm -fr /var/pgsql/data/serializable/*
initdb /var/pgsql/data/serializable
/usr/local/pgsql-serializable/pg_ctl \ -D /var/pgsql/data/serializable -l logfile start
make installcheck
# wait 30 to 60 seconds
/usr/local/pgsql-serializable/pg_ctl \ -D /var/pgsql/data/serializable -m immediate stop
/usr/local/pgsql-serializable/pg_ctl \ -D /var/pgsql/data/serializable -l logfile start
The postmaster is not running.  The tail of logfile is:
LOG:  wait_for_stats delayed 0.013267 seconds
STATEMENT:  SELECT wait_for_stats();
LOG:  received immediate shutdown request
WARNING:  terminating connection because of crash of another server
process
DETAIL:  The postmaster has commanded this server process to roll back
the current transaction and exit, because another server process exited
abnormally and possibly corrupted shared memory.
HINT:  In a moment you should be able to reconnect to the database and
repeat your command.
LOG:  database system was interrupted; last known up at 2010-08-30
09:13:23 CDT
LOG:  database system was not properly shut down; automatic recovery in
progress
LOG:  consistent recovery state reached at 0/5C5D04
LOG:  redo starts at 0/5C5D04
TRAP: FailedAssertion("!(backend == (-1))", File: "catalog.c", Line:
120)
LOG:  startup process (PID 5338) was terminated by signal 6: Aborted
LOG:  aborting startup due to startup process failure
Joe found this while beating up on the SSI patch, but I'm also
seeing it on unpatched HEAD, as checked out from git.  This is 32
bit kubuntu and ubuntu (different versions).
-Kevin

"Kevin Grittner" <Kevin.Grittner@wicourts.gov> writes:
> Checking out HEAD from the git repository I see an assertion
> failure.

The buildfarm isn't reporting any such thing.  Could you get a CVS
checkout and diff it against the git results?
        regards, tom lane

Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Could you get a CVS checkout and diff it against the git results?
Will do.
-Kevin

On 08/30/2010 08:16 AM, Tom Lane wrote:
> "Kevin Grittner" <Kevin.Grittner@wicourts.gov> writes:
>> Checking out HEAD from the git repository I see an assertion
>> failure.
>
> The buildfarm isn't reporting any such thing.  Could you get a CVS
> checkout and diff it against the git results?

I'm seeing it on a checkout from CVS head. Basically do
makemake installinitdbpg_ctl startmake installcheckpg_ctl -m immediate stoppg_ctl start

At this point I see the assertion in the logs.

Joe

--
Joe Conway
credativ LLC: http://www.credativ.us
Linux, PostgreSQL, and general Open Source
Training, Service, Consulting, & 24x7 Support

Joe Conway <mail@joeconway.com> writes:
> I'm seeing it on a checkout from CVS head. Basically do

Oh, OK, I misread Kevin to say this would happen during make
installcheck itself.  So the lack of buildfarm reports is not relevant.

Will get some caffeine in me and then take a look.
        regards, tom lane

"Kevin Grittner" <Kevin.Grittner@wicourts.gov> writes:
> LOG:  database system was interrupted; last known up at 2010-08-30
> 09:13:23 CDT
> LOG:  database system was not properly shut down; automatic recovery in
> progress
> LOG:  consistent recovery state reached at 0/5C5D04
> LOG:  redo starts at 0/5C5D04
> TRAP: FailedAssertion("!(backend == (-1))", File: "catalog.c", Line:
> 120)
> LOG:  startup process (PID 5338) was terminated by signal 6: Aborted
> LOG:  aborting startup due to startup process failure

I can reproduce this; looks like it is fallout from the
RelFileNodeBackend patch.  Stack trace from core dump is

#4  0x4f912c in ExceptionalCondition (   conditionName=0x89054 "!(backend == (-1))",    errorType=0x89044
"FailedAssertion",fileName=0x88fc8 "catalog.c",    lineNumber=120) at assert.c:57
 
#5  0x2473b8 in relpathbackend (rnode={spcNode = 1664, dbNode = 0,      relNode = 11617}, backend=2063810256,
forknum=2063672808)  at catalog.c:120
 
#6  0x40a628 in mdopen (reln=0x40096748, forknum=VISIBILITYMAP_FORKNUM,    behavior=EXTENSION_RETURN_NULL) at md.c:508
#7  0x409ce0 in mdexists (reln=0x40096748, forkNum=VISIBILITYMAP_FORKNUM)   at md.c:237
#8  0x40c7ac in smgrexists (reln=0x7b011c28, forknum=2063848444) at smgr.c:207
#9  0x1f2464 in vm_readbuf (rel=0x40061bc0, blkno=0, extend=0 '\000')   at visibilitymap.c:410
#10 0x1f1b80 in visibilitymap_clear (rel=0x7b011c28, heapBlk=2063848444)   at visibilitymap.c:147
#11 0x1e924c in heap_xlog_insert (lsn={xlogid = 0, xrecoff = 26015704},    record=0x30) at heapam.c:4389
#12 0x1eaec8 in heap_redo (lsn={xlogid = 0, xrecoff = 26015704},    record=0x40083f70) at heapam.c:4823
#13 0x21bbd4 in StartupXLOG () at xlog.c:6229
#14 0x2215f8 in StartupProcessMain () at xlog.c:9233
#15 0x24571c in AuxiliaryProcessMain (argc=2, argv=0x7b03ade8)   at bootstrap.c:412
#16 0x3cb560 in StartChildProcess (type=StartupProcess) at postmaster.c:4407
#17 0x3c6f6c in PostmasterMain (argc=3, argv=0x7b03aba0) at postmaster.c:1089
#18 0x34d998 in main (argc=3, argv=0x7b03aba0) at main.c:188

I guess that something isn't properly setting up rnode.backend in
recovery processing, but didn't find it yet.
        regards, tom lane

I wrote:
> I guess that something isn't properly setting up rnode.backend in
> recovery processing, but didn't find it yet.

CreateFakeRelcacheEntry is the culprit ...
        regards, tom lane

On Mon, Aug 30, 2010 at 12:47 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> I wrote:
>> I guess that something isn't properly setting up rnode.backend in
>> recovery processing, but didn't find it yet.
>
> CreateFakeRelcacheEntry is the culprit ...

Thanks for the fix.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company