Re: [HACKERS] IndexBuild Function call fcinfo cannot access memory

Поиск
Список
Период
Сортировка
От Jia Yu
Тема Re: [HACKERS] IndexBuild Function call fcinfo cannot access memory
Дата
Msg-id CAN7JVgWYq+EtHvxn=y0w9Y=baMGhcC5cW2Hfk2_BYL1YDc-jig@mail.gmail.com
обсуждение исходный текст
Ответ на [HACKERS] IndexBuild Function call fcinfo cannot access memory  (Jia Yu <jiayu198910@gmail.com>)
Список pgsql-hackers
Дерево обсуждения
Dear Hackers,

I have solved this problem. I just want to post my answer here for people who want to write their own backend index access method.

When we write a backend index access method (Let's call this method ABC), we need to implement two interfaces: ABCbuild, ABCbuildcallback.

ABCbuild is the main index initialization function and it will make some preparation and call (not directly call, ask Postgres to call) ABCbuildcallback function per each parent table tuple. For a given parent table tuple, ABCbuildcallback will generate the corresponding index entry and insert it into the index main body.

When all parent table tuples have been processed by ABCbuildcallback, Postgres will return the control back to ABCbuild.

As we can see, ABCbuild and ABCbuildcallback need to communicate with each other to transfer some information. This information is stored in a data structure called Struct ABCbuildstate. This ABCbuildstate is first initialized in ABCbuild and then passed to ABCbuildcallback. NOTE THAT: this Struct ABCbuildstate has to be initialized by palloc() in ABCbuild because ABCbuild is not directly calling ABCbuildcallback. My mistake is that I initialize/declare this ABCbuildstate struct by using regular C like this "ABCbuildstate my ABCbuildstate;" This will lead to segmentation fault sent by OS. However, this wrong declare method still works when we add some unit SQL script in Postgres regression test (execute "make check"). We still are able to build ABC index "create index ABC_idx on ... using ABC" in Postgres regression test script. But it will make us fail at normal Postgres server and general debug methods, such as set breakpoint or print debug information, cannot find this error because they even cannot reach the ABCbuild function.

I believe there are some special settings in Postgres regression test program "src/test/regress/pg_regress.c" but I haven't found it. These special settings will make Postgres regression test tolerates some memory errors. 


Hope this will help you!

Thanks,
Jia Yu


On Mon, Jan 23, 2017 at 1:40 PM, Jia Yu <jiayu198910@gmail.com> wrote:
Dear hackers,

Currently, I am developing a backend index access method for my research project.

I built corresponding routines such as MyIndexbuild, MyIndexInsert, and so on and put them in "src/backend/access/hippo" (hippo is my index's name). I also added new entries in corresponding catalog files:

src/include/catalog/pg_am.h
src/include/catalog/pg_amop.h
src/include/catalog/pg_amproc.h
src/include/catalog/pg_opclass.h
src/include/catalog/pg_opfamily.h
src/include/catalog/pg_proc.h

Now I can successfully build/query/update my new index by executing PG source code regression test (make check). Everything looks fine.


However, these methods don't work in the normal PG server. It gave me "segmentation fault"

I entered this SQL command in psql:
create index hippo_idx on hippo_tbl using hippo(id2);

Then got:
Segmentation fault (core dumped)

Here is my backtrace. It looks like I cannot access fcinfo. Can you help me about this? Or just some hints? I have been struggling with this problem for weeks. 

Thank you all in advance!

----------------------------------------
#0  0x00000000004673d2 in hippobuild (
    fcinfo=<error reading variable: Cannot access memory at address 0x7fff3092c188>) at hippo.c:154
#1  0x0000000000956e64 in OidFunctionCall3Coll (functionId=5005, collation=0, 
    arg1=140162511383536, arg2=140162511390256, arg3=19885792) at fmgr.c:1649
#2  0x0000000000555809 in index_build (heapRelation=0x7f7a20b3abf0, 
    indexRelation=0x7f7a20b3c630, indexInfo=0x12f6ee0, isprimary=0 '\000', 
    isreindex=0 '\000') at index.c:2025
#3  0x0000000000554551 in index_create (heapRelation=0x7f7a20b3abf0, 
    indexRelationName=0x12f9378 "hippo_idx", indexRelationId=20668, 
    relFileNode=0, indexInfo=0x12f6ee0, indexColNames=0x12f7370, 
    accessMethodObjectId=5000, tableSpaceId=0, collationObjectId=0x12f7990, 
    classObjectId=0x12f79b0, coloptions=0x12f79d0, reloptions=19895952, 
    isprimary=0 '\000', isconstraint=0 '\000', deferrable=0 '\000', 
    initdeferred=0 '\000', allow_system_table_mods=0 '\000', 
    skip_build=0 '\000', concurrent=0 '\000', is_internal=0 '\000', 
    if_not_exists=0 '\000') at index.c:1100
#4  0x000000000062727b in DefineIndex (relationId=12476, stmt=0x12f6ff8, 
    indexRelationId=0, is_alter_table=0 '\000', check_rights=1 '\001', 
    skip_build=0 '\000', quiet=0 '\000') at indexcmds.c:606
#5  0x00000000008077a6 in ProcessUtilitySlow (parsetree=0x127c740, 
    queryString=0x127ba48 "create index hippo_idx on hippo_tbl using hippo(id2) with (density=20);\n", context=PROCESS_UTILITY_TOPLEVEL, params=0x0, 
    dest=0xe19a40 <debugtupDR>, completionTag=0x7fff7819aae0 "")
    at utility.c:1260
#6  0x0000000000806c30 in standard_ProcessUtility (parsetree=0x127c740, 
    queryString=0x127ba48 "create index hippo_idx on hippo_tbl using hippo(id2) with (density=20);\n", context=PROCESS_UTILITY_TOPLEVEL, params=0x0, 
    dest=0xe19a40 <debugtupDR>, completionTag=0x7fff7819aae0 "")
    at utility.c:893
#7  0x0000000000805d81 in ProcessUtility (parsetree=0x127c740, 
    queryString=0x127ba48 "create index hippo_idx on hippo_tbl using hippo(id2) with (density=20);\n", context=PROCESS_UTILITY_TOPLEVEL, params=0x0, 
    dest=0xe19a40 <debugtupDR>, completionTag=0x7fff7819aae0 "")
    at utility.c:335
#8  0x0000000000804dd3 in PortalRunUtility (portal=0x12f0d88, 
    utilityStmt=0x127c740, isTopLevel=1 '\001', dest=0xe19a40 <debugtupDR>, 
    completionTag=0x7fff7819aae0 "") at pquery.c:1187
#9  0x0000000000804fa1 in PortalRunMulti (portal=0x12f0d88, 
    isTopLevel=1 '\001', dest=0xe19a40 <debugtupDR>, 
    altdest=0xe19a40 <debugtupDR>, completionTag=0x7fff7819aae0 "")
    at pquery.c:1318
#10 0x00000000008044e5 in PortalRun (portal=0x12f0d88, 
    count=9223372036854775807, isTopLevel=1 '\001', 
    dest=0xe19a40 <debugtupDR>, altdest=0xe19a40 <debugtupDR>, 
    completionTag=0x7fff7819aae0 "") at pquery.c:816
#11 0x00000000007fe2af in exec_simple_query (
    query_string=0x127ba48 "create index hippo_idx on hippo_tbl using hippo(id2) with (density=20);\n") at postgres.c:1104
#12 0x00000000008025af in PostgresMain (argc=6, argv=0x1244a40, 
    dbname=0x124c3a0 "postgres", username=0x124c3a0 "postgres")
    at postgres.c:4025
#13 0x00000000006d3513 in main (argc=6, argv=0x1244a40) at main.c:219







В списке pgsql-hackers по дате отправления:

Предыдущее
От: "David G. Johnston"
Дата:
Сообщение: Re: [HACKERS] [PATCH] Rename pg_switch_xlog to pg_switch_wal
Следующее
От: Andres Freund
Дата:
Сообщение: Re: [HACKERS] [PATCH] Rename pg_switch_xlog to pg_switch_wal