Обсуждение: BUG #16050: Server crash on CREATE TEXT SEARCH DICTIONARY with a wrong AffFile

The following bug has been logged on the website:

Bug reference:      16050
Logged by:          Alexander Lakhin
Email address:      exclusion@gmail.com
PostgreSQL version: 12.0
Operating system:   Ubuntu 18.04
Description:

The following query:
CREATE TEXT SEARCH DICTIONARY hunspell_num (Template=ispell,
DictFile=hunspell_sample_num, AffFile=hunspell_sample_long);

crashes postgres with the stack trace:
Core was generated by `postgres: law regression [local] CREATE TEXT SEARCH
DICTIONARY                '.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x000056512c61b72e in getCompoundAffixFlagValue
(Conf=Conf@entry=0x56512e9d9f70, s=<optimized out>) at spell.c:1126
1126            while (*flagcur)
(gdb) bt
#0  0x000056512c61b72e in getCompoundAffixFlagValue
(Conf=Conf@entry=0x56512e9d9f70, s=<optimized out>) at spell.c:1126
#1  0x000056512c61c00b in makeCompoundFlags (Conf=Conf@entry=0x56512e9d9f70,
affix=<optimized out>) at spell.c:1608
#2  0x000056512c61c16e in mkSPNode (Conf=Conf@entry=0x56512e9d9f70,
low=low@entry=0, high=high@entry=1, 
    level=level@entry=3) at spell.c:1680
#3  0x000056512c61c22b in mkSPNode (Conf=Conf@entry=0x56512e9d9f70,
low=low@entry=0, high=high@entry=1, 
    level=level@entry=2) at spell.c:1692
#4  0x000056512c61c116 in mkSPNode (Conf=Conf@entry=0x56512e9d9f70,
low=low@entry=0, high=high@entry=4, 
    level=level@entry=1) at spell.c:1652
#5  0x000056512c61c116 in mkSPNode (Conf=Conf@entry=0x56512e9d9f70,
low=low@entry=0, high=9, level=level@entry=0)
    at spell.c:1652
#6  0x000056512c61d9eb in NISortDictionary (Conf=Conf@entry=0x56512e9d9f70)
at spell.c:1785
#7  0x000056512c61936d in dispell_init (fcinfo=<optimized out>) at
dict_ispell.c:89
#8  0x000056512c73c7c9 in FunctionCall1Coll
(flinfo=flinfo@entry=0x7ffc6b41a860, collation=collation@entry=0, 
    arg1=arg1@entry=94906674394208) at fmgr.c:1140
#9  0x000056512c73cffb in OidFunctionCall1Coll
(functionId=functionId@entry=3731, collation=collation@entry=0, 
    arg1=94906674394208) at fmgr.c:1418
#10 0x000056512c46c88b in verify_dictoptions (tmplId=tmplId@entry=3733,
dictoptions=<optimized out>, 
    dictoptions@entry=0x56512e9d59e8) at tsearchcmds.c:402
#11 0x000056512c46e6b1 in DefineTSDictionary (names=<optimized out>,
parameters=<optimized out>) at tsearchcmds.c:463
#12 0x000056512c610c6d in ProcessUtilitySlow
(pstate=pstate@entry=0x56512e9d32f8, pstmt=pstmt@entry=0x56512e9b3a08, 
    queryString=queryString@entry=0x56512e9b2628 "CREATE TEXT SEARCH
DICTIONARY hunspell_num (Template=ispell, DictFile=hunspell_sample_num,
AffFile=hunspell_sample_long);",
context=context@entry=PROCESS_UTILITY_TOPLEVEL, 
    params=params@entry=0x0, queryEnv=queryEnv@entry=0x0,
dest=0x56512e9b3b00, completionTag=0x7ffc6b41af50 "")
    at utility.c:1272
#13 0x000056512c61042e in standard_ProcessUtility (pstmt=0x56512e9b3a08, 
    queryString=0x56512e9b2628 "CREATE TEXT SEARCH DICTIONARY hunspell_num
(Template=ispell, DictFile=hunspell_sample_num,
AffFile=hunspell_sample_long);", context=PROCESS_UTILITY_TOPLEVEL,
params=0x0, queryEnv=0x0, dest=0x56512e9b3b00, 
    completionTag=0x7ffc6b41af50 "") at utility.c:927
#14 0x000056512c6104dc in ProcessUtility (pstmt=pstmt@entry=0x56512e9b3a08,
queryString=<optimized out>, 
    context=context@entry=PROCESS_UTILITY_TOPLEVEL, params=<optimized out>,
queryEnv=<optimized out>, 
    dest=dest@entry=0x56512e9b3b00, completionTag=0x7ffc6b41af50 "") at
utility.c:360
#15 0x000056512c60c983 in PortalRunUtility
(portal=portal@entry=0x56512ea198d8, pstmt=pstmt@entry=0x56512e9b3a08, 
    isTopLevel=isTopLevel@entry=true,
setHoldSnapshot=setHoldSnapshot@entry=false, dest=dest@entry=0x56512e9b3b00,

    completionTag=completionTag@entry=0x7ffc6b41af50 "") at pquery.c:1175
#16 0x000056512c60d5d2 in PortalRunMulti
(portal=portal@entry=0x56512ea198d8, isTopLevel=isTopLevel@entry=true, 
    setHoldSnapshot=setHoldSnapshot@entry=false,
dest=dest@entry=0x56512e9b3b00, altdest=altdest@entry=0x56512e9b3b00, 
    completionTag=completionTag@entry=0x7ffc6b41af50 "") at pquery.c:1321
#17 0x000056512c60e342 in PortalRun (portal=portal@entry=0x56512ea198d8,
count=count@entry=9223372036854775807, 
    isTopLevel=isTopLevel@entry=true, run_once=run_once@entry=true,
dest=dest@entry=0x56512e9b3b00, 
    altdest=altdest@entry=0x56512e9b3b00, completionTag=0x7ffc6b41af50 "")
at pquery.c:796
#18 0x000056512c60a5f2 in exec_simple_query (
    query_string=query_string@entry=0x56512e9b2628 "CREATE TEXT SEARCH
DICTIONARY hunspell_num (Template=ispell, DictFile=hunspell_sample_num,
AffFile=hunspell_sample_long);") at postgres.c:1215
#19 0x000056512c60c5c2 in PostgresMain (argc=<optimized out>,
argv=argv@entry=0x56512e9dd9d8, dbname=<optimized out>, 
    username=<optimized out>) at postgres.c:4236
#20 0x000056512c57ea53 in BackendRun (port=port@entry=0x56512e9d60b0) at
postmaster.c:4437
#21 0x000056512c581d19 in BackendStartup (port=port@entry=0x56512e9d60b0) at
postmaster.c:4128
#22 0x000056512c582030 in ServerLoop () at postmaster.c:1704
#23 0x000056512c583421 in PostmasterMain (argc=3, argv=<optimized out>) at
postmaster.c:1377
#24 0x000056512c4dec13 in main (argc=3, argv=0x56512e9aca00) at main.c:228

On Fri, Oct 11, 2019 at 03:05:38PM +0000, PG Bug reporting form wrote:
>The following bug has been logged on the website:
>
>Bug reference:      16050
>Logged by:          Alexander Lakhin
>Email address:      exclusion@gmail.com
>PostgreSQL version: 12.0
>Operating system:   Ubuntu 18.04
>Description:
>
>The following query:
>CREATE TEXT SEARCH DICTIONARY hunspell_num (Template=ispell,
>DictFile=hunspell_sample_num, AffFile=hunspell_sample_long);
>
>crashes postgres with the stack trace:
>Core was generated by `postgres: law regression [local] CREATE TEXT SEARCH
>DICTIONARY                '.

Yep, I can reproduce it quite easily. With extra debug symbols and
memory randomization it produces a bit clearer backtrace:


Program received signal SIGSEGV, Segmentation fault.
0x00000000008fd31b in getCompoundAffixFlagValue (Conf=0x2d053c8, s=0x7f7f7f7f7f7f7f7f <error: Cannot access memory at
address0x7f7f7f7f7f7f7f7f>) at spell.c:1126
 
1126        while (*flagcur)
(gdb) bt
#0  0x00000000008fd31b in getCompoundAffixFlagValue (Conf=0x2d053c8, s=0x7f7f7f7f7f7f7f7f <error: Cannot access memory
ataddress 0x7f7f7f7f7f7f7f7f>) at spell.c:1126
 
#1  0x00000000008fe627 in makeCompoundFlags (Conf=0x2d053c8, affix=303) at spell.c:1608
#2  0x00000000008fe959 in mkSPNode (Conf=0x2d053c8, low=0, high=1, level=3) at spell.c:1680
#3  0x00000000008fea1e in mkSPNode (Conf=0x2d053c8, low=0, high=1, level=2) at spell.c:1692
#4  0x00000000008fe794 in mkSPNode (Conf=0x2d053c8, low=0, high=4, level=1) at spell.c:1652
#5  0x00000000008fe794 in mkSPNode (Conf=0x2d053c8, low=0, high=9, level=0) at spell.c:1652
...

That is, makeCompontFlags calls getCompoundAffixFlagValue with invalid
pointer 's', likely after it got already pfreed.

cheers

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services 

On Fri, Oct 11, 2019 at 07:27:39PM +0200, Tomas Vondra wrote:
>On Fri, Oct 11, 2019 at 03:05:38PM +0000, PG Bug reporting form wrote:
>>The following bug has been logged on the website:
>>
>>Bug reference:      16050
>>Logged by:          Alexander Lakhin
>>Email address:      exclusion@gmail.com
>>PostgreSQL version: 12.0
>>Operating system:   Ubuntu 18.04
>>Description:
>>
>>The following query:
>>CREATE TEXT SEARCH DICTIONARY hunspell_num (Template=ispell,
>>DictFile=hunspell_sample_num, AffFile=hunspell_sample_long);
>>
>>crashes postgres with the stack trace:
>>Core was generated by `postgres: law regression [local] CREATE TEXT SEARCH
>>DICTIONARY                '.
>
>Yep, I can reproduce it quite easily. With extra debug symbols and
>memory randomization it produces a bit clearer backtrace:
>
>
>Program received signal SIGSEGV, Segmentation fault.
>0x00000000008fd31b in getCompoundAffixFlagValue (Conf=0x2d053c8, s=0x7f7f7f7f7f7f7f7f <error: Cannot access memory at
address0x7f7f7f7f7f7f7f7f>) at spell.c:1126
 
>1126        while (*flagcur)
>(gdb) bt
>#0  0x00000000008fd31b in getCompoundAffixFlagValue (Conf=0x2d053c8, s=0x7f7f7f7f7f7f7f7f <error: Cannot access memory
ataddress 0x7f7f7f7f7f7f7f7f>) at spell.c:1126
 
>#1  0x00000000008fe627 in makeCompoundFlags (Conf=0x2d053c8, affix=303) at spell.c:1608
>#2  0x00000000008fe959 in mkSPNode (Conf=0x2d053c8, low=0, high=1, level=3) at spell.c:1680
>#3  0x00000000008fea1e in mkSPNode (Conf=0x2d053c8, low=0, high=1, level=2) at spell.c:1692
>#4  0x00000000008fe794 in mkSPNode (Conf=0x2d053c8, low=0, high=4, level=1) at spell.c:1652
>#5  0x00000000008fe794 in mkSPNode (Conf=0x2d053c8, low=0, high=9, level=0) at spell.c:1652
>...
>
>That is, makeCompontFlags calls getCompoundAffixFlagValue with invalid
>pointer 's', likely after it got already pfreed.
>

FWIW this is a pre-existing bug, it's not new in 12. I've been able to
reproduce it on all releases since 9.6.

On 9.5 the example does not work, because it does not include the
hunspell dictionaries. After copying those from 9.6, the CREATE command
complains about long flags not being supported - I've removed the first
line 'FLAG long' from the affix file, and then it works without a crash.

Looking at the commit history of spell.c, there seems to be a bunch of
commits in 2016 (e.g. f4ceed6ceba3) touching exactly this part of the
code (hunspell), and it also correlates quite nicely with the affected
branches (9.6+). So I guess it's a bug in those changes.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services