Обсуждение: Memory leak in RelationBuildRowSecurity

Hi,

One of our customers faced with backend memory bloat (up to 70Gb).
This customer is actively usnig row based security and temporary tables.
Looks like this combination cause memory bloat.

There is memory leak in RelationBuildRowSecurity function - it converts 
node to expression tree in current memory context:

             with_check_qual = (Expr *) stringToNode(with_check_value);
             qual_expr = (Expr *) stringToNode(qual_value);

then this trees are copied to "row security descriptor" memory context:

             policy->with_check_qual = copyObject(with_check_qual);

and finally we do cleanup:

             /* clean up some (not all) of the junk ... */
             if (qual_expr != NULL)
                 pfree(qual_expr);
             if (with_check_qual != NULL)
                 pfree(with_check_qual);

The problem is that pfree deletes only top element, but not the whole tree.
As far as we do not have deleteObject function, the only solution I see 
is to create this tree in separate memory context.
But RelationBuildRowSecurity has created its own context.
I wonder if it is enough just to switch to this context earlier? 
(proposed patch is attached).

Now a little bit more about the role temporary tables in this problem.
This the stack trace I observed in debugger:

(gdb) bt
#0  AllocSetAlloc (context=0x5634b7389fb8, size=<optimized out>) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/mmgr/aset.c:891
#1  0x00005634b6c348c1 in MemoryContextAllocZero (context=<optimized 
out>, size=size@entry=28) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/mmgr/mcxt.c:737
#2  0x00005634b6a66ddd in _readParam () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/readfuncs.c:492
#3  0x00005634b6a68f75 in parseNodeString () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/readfuncs.c:1417
#4  0x00005634b6a6bc65 in nodeRead (
     token=0x5634d39034bb "{PARAM :paramkind 2 :paramid 1 :paramtype 
1700 :paramtypmod 1048580 :paramcollid 0 :location -1}) :location 112} 
:operName (\"=\") :subselect {QUERY :commandType 1 :querySource 0 
:canSetTag true :utilit"..., tok_len=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/read.c:296
#5  0x00005634b6a6bc17 in nodeRead (token=<optimized out>, 
token@entry=0x0, tok_len=1, tok_len@entry=0) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/read.c:362
#6  0x00005634b6a67119 in _readOpExpr () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/readfuncs.c:651
#7  0x00005634b6a6a4e5 in parseNodeString () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/readfuncs.c:1431
#8  0x00005634b6a6bc65 in nodeRead (
     token=0x5634d39033d5 "{OPEXPR :opno 1752 :opfuncid 1718 
:opresulttype 16 :opretset false :opcollid 0 :inputcollid 0 :args ({VAR 
:varno 1 :varattno 14 :vartype 1700 :vartypmod 1048580 :varcollid 0 
:varlevelsup 0 :varnoold 1"..., token@entry=0x0, tok_len=1, 
tok_len@entry=0) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/read.c:296
#9  0x00005634b6a6aa17 in _readSubLink () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/readfuncs.c:784
#10 parseNodeString () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/readfuncs.c:1441
#11 0x00005634b6a6bc65 in nodeRead (
     token=0x5634d39033a6 "{SUBLINK :subLinkType 2 :subLinkId 0 
:testexpr {OPEXPR :opno 1752 :opfuncid 1718 :opresulttype 16 :opretset 
false :opcollid 0 :inputcollid 0 :args ({VAR :varno 1 :varattno 14 
:vartype 1700 :vartypmod "..., tok_len=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/read.c:296
#12 0x00005634b6a6bc17 in nodeRead (token=<optimized out>, 
token@entry=0x0, tok_len=1, tok_len@entry=0) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/read.c:362
#13 0x00005634b6a69b46 in _readBoolExpr () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/readfuncs.c:768
#14 parseNodeString () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/readfuncs.c:1439
#15 0x00005634b6a6bc65 in nodeRead (
     token=0x5634d3903389 "{BOOLEXPR :boolop and :args ({SUBLINK 
:subLinkType 2 :subLinkId 0 :testexpr {OPEXPR :opno 1752 :opfuncid 1718 
:opresulttype 16 :opretset false :opcollid 0 :inputcollid 0 :args ({VAR 
:varno 1 :varattno"..., tok_len=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/read.c:296
#16 0x00005634b6a6bc17 in nodeRead (token=<optimized out>, 
token@entry=0x0, tok_len=1, tok_len@entry=0) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/read.c:362
#17 0x00005634b6a69b46 in _readBoolExpr () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/readfuncs.c:768
#18 parseNodeString () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/readfuncs.c:1439
#19 0x00005634b6a6bc65 in nodeRead (
     token=0x5634d39032d8 "{BOOLEXPR :boolop and :args ({CONST 
:consttype 16 :consttypmod -1 :constcollid 0 :constlen 1 :constbyval 
true :constisnull false :location 89 :constvalue 1 [ 1 0 0 0 0 0 0 0 ]} 
{BOOLEXPR :boolop and :"..., token@entry=0x0, tok_len=1, 
tok_len@entry=0) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/read.c:296
#20 0x00005634b6a6bf00 in stringToNode (str=<optimized out>) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/read.c:53
#21 0x00005634b69d18dd in RelationBuildRowSecurity 
(relation=relation@entry=0x7f2b5dcee8c8) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/commands/policy.c:281
#22 0x00005634b6c03e56 in RelationBuildDesc 
(targetRelId=targetRelId@entry=158077, insertIt=insertIt@entry=0 '\000')
     at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1052
#23 0x00005634b6c042cf in RelationClearRelation 
(relation=0x7f2b5dce3f28, rebuild=rebuild@entry=1 '\001') at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2200
#24 0x00005634b6c04f95 in RelationCacheInvalidate () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2500
#25 0x00005634b6bfdecf in InvalidateSystemCaches () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:628
#26 0x00005634b6afca8f in ReceiveSharedInvalidMessages 
(invalFunction=invalFunction@entry=0x5634b6bfe6d0 
<LocalExecuteInvalidationMessage>,
     resetFunction=resetFunction@entry=0x5634b6bfdeb0 
<InvalidateSystemCaches>) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/ipc/sinval.c:108
#27 0x00005634b6bfe018 in AcceptInvalidationMessages () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:662
#28 0x00005634b6affb79 in LockRelationOid (relid=2601, lockmode=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/lmgr/lmgr.c:125
#29 0x00005634b68d23b5 in relation_open (relationId=2601, 
lockmode=<optimized out>) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1068
#30 0x00005634b68d264e in heap_open (relationId=<optimized out>, 
lockmode=lockmode@entry=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1246
#31 0x00005634b6bfc957 in SearchCatCache (cache=0x5634b72fb138, 
v1=<optimized out>, v2=v2@entry=0, v3=v3@entry=0, v4=v4@entry=0)
     at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/catcache.c:1239
#32 0x00005634b6c0964e in SearchSysCache (cacheId=cacheId@entry=2, 
key1=<optimized out>, key2=key2@entry=0, key3=key3@entry=0, 
key4=key4@entry=0)
     at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/syscache.c:981
#33 0x00005634b6c02940 in RelationInitIndexAccessInfo 
(relation=relation@entry=0x7f2946a9dfa8) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1211
---Type <return> to continue, or q <return> to quit---
#34 0x00005634b6c03e6d in RelationBuildDesc (targetRelId=<optimized
out>, insertIt=insertIt@entry=1 '\001') at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1060
#35 0x00005634b6c04d05 in RelationIdGetRelation 
(relationId=relationId@entry=2656) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1782
#36 0x00005634b68d236f in relation_open 
(relationId=relationId@entry=2656, lockmode=lockmode@entry=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1071
#37 0x00005634b68e3c6e in index_open (relationId=relationId@entry=2656, 
lockmode=lockmode@entry=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/index/indexam.c:167
#38 0x00005634b68e36e3 in systable_beginscan 
(heapRelation=heapRelation@entry=0x7f2946a32ce8, 
indexId=indexId@entry=2656, indexOK=indexOK@entry=1 '\001', 
snapshot=snapshot@entry=0x0,
     nkeys=nkeys@entry=1, key=key@entry=0x7ffed5111320) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/index/genam.c:335
#39 0x00005634b6c03736 in AttrDefaultFetch (relation=0x7f2b5dc97258) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:3655
#40 RelationBuildTupleDesc (relation=0x7f2b5dc97258) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:607
#41 RelationBuildDesc (targetRelId=targetRelId@entry=158077, 
insertIt=insertIt@entry=0 '\000') at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1033
#42 0x00005634b6c042cf in RelationClearRelation 
(relation=0x7f2b5dce3f28, rebuild=rebuild@entry=1 '\001') at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2200
#43 0x00005634b6c04f95 in RelationCacheInvalidate () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2500
#44 0x00005634b6bfdecf in InvalidateSystemCaches () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:628
#45 0x00005634b6afca8f in ReceiveSharedInvalidMessages 
(invalFunction=invalFunction@entry=0x5634b6bfe6d0 
<LocalExecuteInvalidationMessage>,
     resetFunction=resetFunction@entry=0x5634b6bfdeb0 
<InvalidateSystemCaches>) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/ipc/sinval.c:108
#46 0x00005634b6bfe018 in AcceptInvalidationMessages () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:662
#47 0x00005634b6affb79 in LockRelationOid (relid=2679, lockmode=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/lmgr/lmgr.c:125
#48 0x00005634b68d23b5 in relation_open 
(relationId=relationId@entry=2679, lockmode=lockmode@entry=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1068
#49 0x00005634b68e3c6e in index_open (relationId=relationId@entry=2679, 
lockmode=lockmode@entry=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/index/indexam.c:167
#50 0x00005634b68e36e3 in systable_beginscan 
(heapRelation=heapRelation@entry=0x7f2946aa0748, indexId=2679, 
indexOK=<optimized out>, snapshot=snapshot@entry=0x0, nkeys=1,
     key=key@entry=0x7ffed5111790) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/index/genam.c:335
#51 0x00005634b6bfc981 in SearchCatCache (cache=0x5634b7301ec8, 
v1=<optimized out>, v2=v2@entry=0, v3=v3@entry=0, v4=v4@entry=0)
     at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/catcache.c:1241
#52 0x00005634b6c0964e in SearchSysCache (cacheId=cacheId@entry=32, 
key1=<optimized out>, key2=key2@entry=0, key3=key3@entry=0, 
key4=key4@entry=0)
     at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/syscache.c:981
#53 0x00005634b6c028d8 in RelationInitIndexAccessInfo 
(relation=relation@entry=0x7f2946969f38) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1197
#54 0x00005634b6c03e6d in RelationBuildDesc (targetRelId=<optimized 
out>, insertIt=insertIt@entry=1 '\001') at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1060
#55 0x00005634b6c04d05 in RelationIdGetRelation 
(relationId=relationId@entry=2656) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1782
#56 0x00005634b68d236f in relation_open 
(relationId=relationId@entry=2656, lockmode=lockmode@entry=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1071
#57 0x00005634b68e3c6e in index_open (relationId=relationId@entry=2656, 
lockmode=lockmode@entry=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/index/indexam.c:167
#58 0x00005634b68e36e3 in systable_beginscan 
(heapRelation=heapRelation@entry=0x7f2946a32ce8, 
indexId=indexId@entry=2656, indexOK=indexOK@entry=1 '\001', 
snapshot=snapshot@entry=0x0,
     nkeys=nkeys@entry=1, key=key@entry=0x7ffed5111bf0) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/index/genam.c:335
#59 0x00005634b6c03736 in AttrDefaultFetch (relation=0x7f2b5dcedf78) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:3655
#60 RelationBuildTupleDesc (relation=0x7f2b5dcedf78) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:607
#61 RelationBuildDesc (targetRelId=targetRelId@entry=158077, 
insertIt=insertIt@entry=0 '\000') at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1033
#62 0x00005634b6c042cf in RelationClearRelation 
(relation=0x7f2b5dce3f28, rebuild=rebuild@entry=1 '\001') at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2200
#63 0x00005634b6c04f95 in RelationCacheInvalidate () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2500
#64 0x00005634b6bfdecf in InvalidateSystemCaches () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:628
#65 0x00005634b6afca8f in ReceiveSharedInvalidMessages 
(invalFunction=invalFunction@entry=0x5634b6bfe6d0 
<LocalExecuteInvalidationMessage>,
     resetFunction=resetFunction@entry=0x5634b6bfdeb0 
<InvalidateSystemCaches>) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/ipc/sinval.c:108
#66 0x00005634b6bfe018 in AcceptInvalidationMessages () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:662
#67 0x00005634b6affb79 in LockRelationOid (relid=2610, lockmode=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/lmgr/lmgr.c:125
#68 0x00005634b68d23b5 in relation_open (relationId=2610, 
lockmode=<optimized out>) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1068
#69 0x00005634b68d264e in heap_open (relationId=<optimized out>, 
lockmode=lockmode@entry=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1246
#70 0x00005634b6bfc957 in SearchCatCache (cache=0x5634b7301ec8, 
v1=<optimized out>, v2=v2@entry=0, v3=v3@entry=0, v4=v4@entry=0)
     at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/catcache.c:1239
#71 0x00005634b6c0964e in SearchSysCache (cacheId=cacheId@entry=32, 
key1=<optimized out>, key2=key2@entry=0, key3=key3@entry=0, 
key4=key4@entry=0)
     at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/syscache.c:981
#72 0x00005634b6c028d8 in RelationInitIndexAccessInfo 
(relation=relation@entry=0x7f2946dcc698) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1197
---Type <return> to continue, or q <return> to quit---
#73 0x00005634b6c03e6d in RelationBuildDesc (targetRelId=<optimized 
out>, insertIt=insertIt@entry=1 '\001') at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1060
#74 0x00005634b6c04d05 in RelationIdGetRelation 
(relationId=relationId@entry=2656) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1782
#75 0x00005634b68d236f in relation_open 
(relationId=relationId@entry=2656, lockmode=lockmode@entry=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1071
#76 0x00005634b68e3c6e in index_open (relationId=relationId@entry=2656, 
lockmode=lockmode@entry=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/index/indexam.c:167
#77 0x00005634b68e36e3 in systable_beginscan 
(heapRelation=heapRelation@entry=0x7f2946a32ce8, 
indexId=indexId@entry=2656, indexOK=indexOK@entry=1 '\001', 
snapshot=snapshot@entry=0x0,
     nkeys=nkeys@entry=1, key=key@entry=0x7ffed5112470) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/index/genam.c:335
#78 0x00005634b6c03736 in AttrDefaultFetch (relation=0x7f2946f4d1f8) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:3655
#79 RelationBuildTupleDesc (relation=0x7f2946f4d1f8) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:607
#80 RelationBuildDesc (targetRelId=targetRelId@entry=158077, 
insertIt=insertIt@entry=0 '\000') at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1033
#81 0x00005634b6c042cf in RelationClearRelation 
(relation=0x7f2b5dce3f28, rebuild=rebuild@entry=1 '\001') at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2200
#82 0x00005634b6c04f95 in RelationCacheInvalidate () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2500
#83 0x00005634b6bfdecf in InvalidateSystemCaches () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:628
#84 0x00005634b6afca8f in ReceiveSharedInvalidMessages 
(invalFunction=invalFunction@entry=0x5634b6bfe6d0 
<LocalExecuteInvalidationMessage>,
     resetFunction=resetFunction@entry=0x5634b6bfdeb0 
<InvalidateSystemCaches>) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/ipc/sinval.c:108
#85 0x00005634b6bfe018 in AcceptInvalidationMessages () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:662
#86 0x00005634b6affb79 in LockRelationOid (relid=2656, lockmode=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/lmgr/lmgr.c:125
#87 0x00005634b68d23b5 in relation_open 
(relationId=relationId@entry=2656, lockmode=lockmode@entry=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1068
#88 0x00005634b68e3c6e in index_open (relationId=relationId@entry=2656, 
lockmode=lockmode@entry=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/index/indexam.c:167
#89 0x00005634b68e36e3 in systable_beginscan 
(heapRelation=heapRelation@entry=0x7f2946a32ce8, 
indexId=indexId@entry=2656, indexOK=indexOK@entry=1 '\001', 
snapshot=snapshot@entry=0x0,
     nkeys=nkeys@entry=1, key=key@entry=0x7ffed51128f0) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/index/genam.c:335
#90 0x00005634b6c03736 in AttrDefaultFetch (relation=0x7f2b5dce79d8) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:3655
#91 RelationBuildTupleDesc (relation=0x7f2b5dce79d8) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:607
#92 RelationBuildDesc (targetRelId=targetRelId@entry=158077, 
insertIt=insertIt@entry=0 '\000') at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1033
#93 0x00005634b6c042cf in RelationClearRelation 
(relation=0x7f2b5dce3f28, rebuild=rebuild@entry=1 '\001') at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2200
#94 0x00005634b6c04f95 in RelationCacheInvalidate () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2500
#95 0x00005634b6bfdecf in InvalidateSystemCaches () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:628
#96 0x00005634b6afca8f in ReceiveSharedInvalidMessages 
(invalFunction=invalFunction@entry=0x5634b6bfe6d0 
<LocalExecuteInvalidationMessage>,
     resetFunction=resetFunction@entry=0x5634b6bfdeb0 
<InvalidateSystemCaches>) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/ipc/sinval.c:108
#97 0x00005634b6bfe018 in AcceptInvalidationMessages () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:662
#98 0x00005634b6affb79 in LockRelationOid (relid=2604, lockmode=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/lmgr/lmgr.c:125
#99 0x00005634b68d23b5 in relation_open (relationId=2604, 
lockmode=<optimized out>) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1068
#100 0x00005634b68d264e in heap_open (relationId=relationId@entry=2604, 
lockmode=lockmode@entry=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1246
#101 0x00005634b6c0370e in AttrDefaultFetch (relation=0x5634b7310cd8) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:3654
#102 RelationBuildTupleDesc (relation=0x5634b7310cd8) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:607
#103 RelationBuildDesc (targetRelId=targetRelId@entry=158077, 
insertIt=insertIt@entry=0 '\000') at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1033
#104 0x00005634b6c042cf in RelationClearRelation 
(relation=0x7f2b5dce3f28, rebuild=rebuild@entry=1 '\001') at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2200
#105 0x00005634b6c04f95 in RelationCacheInvalidate () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2500
#106 0x00005634b6bfdecf in InvalidateSystemCaches () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:628
#107 0x00005634b6afca8f in ReceiveSharedInvalidMessages 
(invalFunction=invalFunction@entry=0x5634b6bfe6d0 
<LocalExecuteInvalidationMessage>,
     resetFunction=resetFunction@entry=0x5634b6bfdeb0 
<InvalidateSystemCaches>) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/ipc/sinval.c:108
#108 0x00005634b6bfe018 in AcceptInvalidationMessages () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:662
#109 0x00005634b6affb79 in LockRelationOid (relid=2659, lockmode=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/lmgr/lmgr.c:125
#110 0x00005634b68d23b5 in relation_open 
(relationId=relationId@entry=2659, lockmode=lockmode@entry=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1068
#111 0x00005634b68e3c6e in index_open (relationId=relationId@entry=2659, 
lockmode=lockmode@entry=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/index/indexam.c:167
#112 0x00005634b68e36e3 in systable_beginscan 
(heapRelation=heapRelation@entry=0x7f2b5dc94f38, 
indexId=indexId@entry=2659, indexOK=<optimized out>, 
snapshot=snapshot@entry=0x0,
     nkeys=nkeys@entry=2, key=key@entry=0x7ffed51131f0) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/index/genam.c:335
#113 0x00005634b6c03536 in RelationBuildTupleDesc 
(relation=0x7f2b5dc991c8) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:514
#114 RelationBuildDesc (targetRelId=targetRelId@entry=158077, 
insertIt=insertIt@entry=0 '\000') at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1033
---Type <return> to continue, or q <return> to quit---
#115 0x00005634b6c042cf in RelationClearRelation 
(relation=0x7f2b5dce3f28, rebuild=rebuild@entry=1 '\001') at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2200
#116 0x00005634b6c04f95 in RelationCacheInvalidate () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2500
#117 0x00005634b6bfdecf in InvalidateSystemCaches () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:628
#118 0x00005634b6afca8f in ReceiveSharedInvalidMessages 
(invalFunction=invalFunction@entry=0x5634b6bfe6d0 
<LocalExecuteInvalidationMessage>,
     resetFunction=resetFunction@entry=0x5634b6bfdeb0 
<InvalidateSystemCaches>) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/ipc/sinval.c:108
#119 0x00005634b6bfe018 in AcceptInvalidationMessages () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:662
#120 0x00005634b6affb79 in LockRelationOid (relid=1249, lockmode=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/lmgr/lmgr.c:125
#121 0x00005634b68d23b5 in relation_open (relationId=1249, 
lockmode=<optimized out>) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1068
#122 0x00005634b68d264e in heap_open (relationId=relationId@entry=1249, 
lockmode=lockmode@entry=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1246
#123 0x00005634b6c03505 in RelationBuildTupleDesc 
(relation=0x7f2946aa0a68) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:513
#124 RelationBuildDesc (targetRelId=targetRelId@entry=158077, 
insertIt=insertIt@entry=0 '\000') at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1033
#125 0x00005634b6c042cf in RelationClearRelation 
(relation=0x7f2b5dce3f28, rebuild=rebuild@entry=1 '\001') at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2200
#126 0x00005634b6c04f95 in RelationCacheInvalidate () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2500
#127 0x00005634b6bfdecf in InvalidateSystemCaches () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:628
#128 0x00005634b6afca8f in ReceiveSharedInvalidMessages 
(invalFunction=invalFunction@entry=0x5634b6bfe6d0 
<LocalExecuteInvalidationMessage>,
     resetFunction=resetFunction@entry=0x5634b6bfdeb0 
<InvalidateSystemCaches>) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/ipc/sinval.c:108
#129 0x00005634b6bfe018 in AcceptInvalidationMessages () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:662
#130 0x00005634b6affb79 in LockRelationOid (relid=2662, lockmode=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/lmgr/lmgr.c:125
#131 0x00005634b68d23b5 in relation_open 
(relationId=relationId@entry=2662, lockmode=lockmode@entry=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1068
#132 0x00005634b68e3c6e in index_open (relationId=relationId@entry=2662, 
lockmode=lockmode@entry=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/index/indexam.c:167
#133 0x00005634b68e36e3 in systable_beginscan 
(heapRelation=heapRelation@entry=0x7f2b5dcf0318, 
indexId=indexId@entry=2662, indexOK=<optimized out>, 
snapshot=0x5634b7048f60,
     nkeys=nkeys@entry=1, key=key@entry=0x7ffed5113a10) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/index/genam.c:335
#134 0x00005634b6c0076e in ScanPgRelation 
(targetRelId=targetRelId@entry=158077, indexOK=indexOK@entry=1 '\001', 
force_non_historic=force_non_historic@entry=0 '\000')
     at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:339
#135 0x00005634b6c032a7 in RelationBuildDesc 
(targetRelId=targetRelId@entry=158077, insertIt=insertIt@entry=0 '\000')
     at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:950
#136 0x00005634b6c042cf in RelationClearRelation 
(relation=0x7f2b5dce3f28, rebuild=rebuild@entry=1 '\001') at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2200
#137 0x00005634b6c04f95 in RelationCacheInvalidate () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2500
#138 0x00005634b6bfdecf in InvalidateSystemCaches () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:628
#139 0x00005634b6afca8f in ReceiveSharedInvalidMessages 
(invalFunction=invalFunction@entry=0x5634b6bfe6d0 
<LocalExecuteInvalidationMessage>,
     resetFunction=resetFunction@entry=0x5634b6bfdeb0 
<InvalidateSystemCaches>) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/ipc/sinval.c:108
#140 0x00005634b6bfe018 in AcceptInvalidationMessages () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:662
#141 0x00005634b6affb79 in LockRelationOid (relid=3256, lockmode=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/lmgr/lmgr.c:125
#142 0x00005634b68d23b5 in relation_open (relationId=3256, 
lockmode=<optimized out>) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1068
#143 0x00005634b68d264e in heap_open (relationId=relationId@entry=3256, 
lockmode=lockmode@entry=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1246
#144 0x00005634b69d1562 in RelationBuildRowSecurity 
(relation=relation@entry=0x7f2946e51c98) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/commands/policy.c:222
#145 0x00005634b6c03e56 in RelationBuildDesc 
(targetRelId=targetRelId@entry=158077, insertIt=insertIt@entry=0 '\000')
     at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1052
#146 0x00005634b6c042cf in RelationClearRelation 
(relation=0x7f2b5dce3f28, rebuild=rebuild@entry=1 '\001') at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2200
#147 0x00005634b6c04f95 in RelationCacheInvalidate () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2500
#148 0x00005634b6bfdecf in InvalidateSystemCaches () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:628
#149 0x00005634b6afca8f in ReceiveSharedInvalidMessages 
(invalFunction=invalFunction@entry=0x5634b6bfe6d0 
<LocalExecuteInvalidationMessage>,
     resetFunction=resetFunction@entry=0x5634b6bfdeb0 
<InvalidateSystemCaches>) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/ipc/sinval.c:108
#150 0x00005634b6bfe018 in AcceptInvalidationMessages () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:662
#151 0x00005634b6affb79 in LockRelationOid (relid=1259, lockmode=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/lmgr/lmgr.c:125
#152 0x00005634b68d23b5 in relation_open (relationId=1259, 
lockmode=<optimized out>) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1068
#153 0x00005634b68d264e in heap_open (relationId=<optimized out>, 
lockmode=lockmode@entry=1) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1246
#154 0x00005634b6bfc957 in SearchCatCache (cache=0x5634b7306b68, 
v1=v1@entry=158077, v2=v2@entry=0, v3=v3@entry=0, v4=v4@entry=0)
---Type <return> to continue, or q <return> to quit---
     at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/catcache.c:1239
#155 0x00005634b6c0964e in SearchSysCache (cacheId=cacheId@entry=45, 
key1=key1@entry=158077, key2=key2@entry=0, key3=key3@entry=0, 
key4=key4@entry=0)
     at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/syscache.c:981
#156 0x00005634b6c31c94 in check_enable_rls (relid=158077, 
checkAsUser=<optimized out>, noError=noError@entry=0 '\000')
     at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/misc/rls.c:66
#157 0x00005634b6ae9d2d in get_row_security_policies 
(root=root@entry=0x5634b7390838, rte=rte@entry=0x5634b7390948, 
rt_index=rt_index@entry=1, 
securityQuals=securityQuals@entry=0x7ffed5114780,
     withCheckOptions=withCheckOptions@entry=0x7ffed5114788, 
hasRowSecurity=hasRowSecurity@entry=0x7ffed511477d "", 
hasSubLinks=hasSubLinks@entry=0x7ffed511477e "")
     at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/rewrite/rowsecurity.c:132
#158 0x00005634b6ae4e4d in fireRIRrules (parsetree=0x5634b7390838, 
activeRIRs=<optimized out>, activeRIRs@entry=0x0, 
forUpdatePushedDown=forUpdatePushedDown@entry=0 '\000')
     at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/rewrite/rewriteHandler.c:1782
#159 0x00005634b6ae7bb4 in QueryRewrite 
(parsetree=parsetree@entry=0x5634b7390838) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/rewrite/rewriteHandler.c:3504
#160 0x00005634b6b152ca in pg_rewrite_query (query=0x5634b7390838) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/tcop/postgres.c:759
#161 0x00005634b6b1556d in pg_analyze_and_rewrite_params 
(parsetree=<optimized out>, query_string=<optimized out>, 
parserSetup=parserSetup@entry=0x5634b6a17cd0 <sql_fn_parser_setup>,
     parserSetupArg=<optimized out>) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/tcop/postgres.c:726
#162 0x00005634b6a19032 in init_sql_fcache (lazyEvalOK=1 '\001', 
collation=<optimized out>, finfo=<optimized out>)
     at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/executor/functions.c:710
#163 fmgr_sql (fcinfo=0x5634b72d2fb8) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/executor/functions.c:1060
#164 0x00005634b6a11617 in ExecMakeFunctionResultNoSets 
(fcache=0x5634b72d2f48, econtext=0x5634b72d2d58, isNull=0x5634b72d38d0 
"Xu5\267\064V", isDone=<optimized out>)
     at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/executor/execQual.c:2020
#165 0x00005634b6a15a02 in ExecTargetList (isDone=0x7ffed5114a4c, 
itemIsDone=0x5634b72d39e8, isnull=0x5634b72d38d0 "Xu5\267\064V", 
values=0x5634b72d38b8, econtext=0x5634b72d2d58,
     targetlist=0x5634b72d39b8, tupdesc=<optimized out>) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/executor/execQual.c:5417
#166 ExecProject (projInfo=<optimized out>, 
isDone=isDone@entry=0x7ffed5114a4c) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/executor/execQual.c:5641
#167 0x00005634b6a2a012 in ExecResult (node=node@entry=0x5634b72d2c48) 
at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/executor/nodeResult.c:155
#168 0x00005634b6a0ea38 in ExecProcNode (node=node@entry=0x5634b72d2c48) 
at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/executor/execProcnode.c:392
#169 0x00005634b6a0b6b6 in ExecutePlan (dest=0x5634b73371c0, 
direction=<optimized out>, numberTuples=0, sendTuples=1 '\001', 
operation=CMD_SELECT, planstate=0x5634b72d2c48,
     estate=0x5634b72d2b38) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/executor/execMain.c:1601
#170 standard_ExecutorRun (queryDesc=0x5634bfe215e8, 
direction=<optimized out>, count=0) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/executor/execMain.c:341
#171 0x00007f2b56a63395 in pgss_ExecutorRun (queryDesc=0x5634bfe215e8, 
direction=ForwardScanDirection, count=0)
     at 
/home/builder/postgrespro-9.5.7.1/build/../contrib/pg_stat_statements/pg_stat_statements.c:878
#172 0x00005634b6b187ff in PortalRunSelect 
(portal=portal@entry=0x5634b7396c18, forward=forward@entry=1 '\001', 
count=0, count@entry=9223372036854775807, dest=dest@entry=0x5634b73371c0)
     at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/tcop/pquery.c:947
#173 0x00005634b6b19c97 in PortalRun 
(portal=portal@entry=0x5634b7396c18, 
count=count@entry=9223372036854775807, isTopLevel=isTopLevel@entry=1 
'\001', dest=dest@entry=0x5634b73371c0,
     altdest=altdest@entry=0x5634b73371c0, 
completionTag=completionTag@entry=0x7ffed5114fe0 "") at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/tcop/pquery.c:791
#174 0x00005634b6b16d12 in exec_simple_query 
(query_string=0x5634b73352a8 "select norders();") at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/tcop/postgres.c:1104
#175 PostgresMain (argc=<optimized out>, argv=argv@entry=0x5634b72b23c0, 
dbname=0x5634b72b2270 "tsup", username=<optimized out>)
     at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/tcop/postgres.c:4102
#176 0x00005634b68a2ab8 in BackendRun (port=0x5634b72e2310) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/postmaster/postmaster.c:4277
#177 BackendStartup (port=0x5634b72e2310) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/postmaster/postmaster.c:3951
#178 ServerLoop () at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/postmaster/postmaster.c:1716
#179 0x00005634b6abad5b in PostmasterMain (argc=argc@entry=3, 
argv=argv@entry=0x5634b72b1380) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/postmaster/postmaster.c:1324
#180 0x00005634b68a3a54 in main (argc=3, argv=0x5634b72b1380) at 
/home/builder/postgrespro-9.5.7.1/build/../src/backend/main/main.c:253


As you can see system catalog updates caused by frequent create/drop 
temp tables lead to nested invocation of InvalidateSystemCaches()
This is itself seems to be bad smell and suspicious behavior. And in 
conjunction with memory leak in RelationBuildRowSecurity it leads to 
backend memory bloat.

I do not know how to fix cache invalidation problem. Global temp tables 
can help here.
But right now I just want to fix the bug with memory leak in 
RelationBuildRowSecurity.

-- 
Konstantin Knizhnik
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Konstantin Knizhnik <k.knizhnik@postgrespro.ru> writes:
> There is memory leak in RelationBuildRowSecurity function - it converts 
> node to expression tree in current memory context:

Hm ... this smells a whole lot like the issue we fixed recently for
partition expressions, namely: what the devil are we doing reloading
these expressions during RelationClearRelation?  We should delay
populating that cache until the value is requested.

            regards, tom lane

On 25.09.2020 16:48, Tom Lane wrote:
> Konstantin Knizhnik <k.knizhnik@postgrespro.ru> writes:
>> There is memory leak in RelationBuildRowSecurity function - it converts
>> node to expression tree in current memory context:
> Hm ... this smells a whole lot like the issue we fixed recently for
> partition expressions, namely: what the devil are we doing reloading
> these expressions during RelationClearRelation?  We should delay
> populating that cache until the value is requested.

Sorry, this stack trace was obtained at 9.6 version of Postgres which 
the customer is using.
So may be the problem with cache invalidation is already fixed.
But the memory leak in RelationBuildRowSecurity is still present.
May be it is not so critical if RelationBuildRowSecurity will not be 
called many times in the same memory context.
But at least calling pfree for a tree is very strange idea: we should 
not do it all or allocate tree in separate memory context.

By the way - it is illustration how "SQL function data" is growing after 
each invocation of RelationBuildRowSecurity:

SQL function data: 57344 total in 3 blocks; 53112 free (278 chunks); 
4232 used
SQL function data: 289213248 total in 3137 blocks; 3281544 free (217 
chunks); 285931704 used
SQL function data: 578434688 total in 6265 blocks; 6581320 free (234 
chunks); 571853368 used
SQL function data: 1148488960 total in 12520 blocks; 4793680 free (17 
chunks); 1143695280 used
SQL function data: 2288597504 total in 25030 blocks; 1222064 free (221 
chunks); 2287375440 used

Konstantin Knizhnik <k.knizhnik@postgrespro.ru> writes:
> On 25.09.2020 16:48, Tom Lane wrote:
>> Hm ... this smells a whole lot like the issue we fixed recently for
>> partition expressions, namely: what the devil are we doing reloading
>> these expressions during RelationClearRelation?  We should delay
>> populating that cache until the value is requested.

> Sorry, this stack trace was obtained at 9.6 version of Postgres which 
> the customer is using.
> So may be the problem with cache invalidation is already fixed.

Not sure if we've changed anything much in that area.  I've occasionally
wondered if we should not increase the size of the sinval message queue,
but it hasn't happened yet.

Anyway, looking more closely at RelationBuildRowSecurity, it doesn't
seem to be doing anything more dangerous than RelationBuildRuleLock
is; it's definitely not risking recursion, as the partition stuff did.
So I take back the idea that we need to postpone it till the data is
referenced.  However, it is leaking memory.

Attached is a proposed rewrite that borrows some ideas from
RelationBuildRuleLock, and also makes use of MemoryContextSetParent
so we don't need a PG_TRY block.  In a very simple test
(using regress_rls_schema.rls_tbl from the regression database),
this doesn't seem to leak any memory in the caller context,
and it also doesn't make the rscxt any bigger than it was before.

> But at least calling pfree for a tree is very strange idea: we should 
> not do it all or allocate tree in separate memory context.

Yeah, that's pretty useless.  But also I think the string forms of
the expressions are leaking as much memory as the tree forms.

This is against HEAD but I think it wouldn't be hard to back-patch.
Are you in a position to see if it fixes the problem in your customer's
environment?

            regards, tom lane

diff --git a/src/backend/commands/policy.c b/src/backend/commands/policy.c
index 4b4e469493..d3f8e8f06c 100644
--- a/src/backend/commands/policy.c
+++ b/src/backend/commands/policy.c
@@ -187,159 +187,139 @@ policy_role_list_to_array(List *roles, int *num_roles)
 /*
  * Load row security policy from the catalog, and store it in
  * the relation's relcache entry.
+ *
+ * Note that caller should have verified that pg_class.relrowsecurity
+ * is true for this relation.
  */
 void
 RelationBuildRowSecurity(Relation relation)
 {
     MemoryContext rscxt;
     MemoryContext oldcxt = CurrentMemoryContext;
-    RowSecurityDesc *volatile rsdesc = NULL;
+    RowSecurityDesc *rsdesc;
+    Relation    catalog;
+    ScanKeyData skey;
+    SysScanDesc sscan;
+    HeapTuple    tuple;
 
     /*
      * Create a memory context to hold everything associated with this
      * relation's row security policy.  This makes it easy to clean up during
-     * a relcache flush.
+     * a relcache flush.  However, to cover the possibility of an error
+     * partway through, we don't make the context long-lived till we're done.
      */
-    rscxt = AllocSetContextCreate(CacheMemoryContext,
+    rscxt = AllocSetContextCreate(CurrentMemoryContext,
                                   "row security descriptor",
                                   ALLOCSET_SMALL_SIZES);
+    MemoryContextCopyAndSetIdentifier(rscxt,
+                                      RelationGetRelationName(relation));
+
+    rsdesc = MemoryContextAllocZero(rscxt, sizeof(RowSecurityDesc));
+    rsdesc->rscxt = rscxt;
 
     /*
-     * Since rscxt lives under CacheMemoryContext, it is long-lived.  Use a
-     * PG_TRY block to ensure it'll get freed if we fail partway through.
+     * Now scan pg_policy for RLS policies associated with this relation.
+     * Because we use the index on (polrelid, polname), we should consistently
+     * visit the rel's policies in name order, at least when system indexes
+     * aren't disabled.  This simplifies equalRSDesc().
      */
-    PG_TRY();
-    {
-        Relation    catalog;
-        ScanKeyData skey;
-        SysScanDesc sscan;
-        HeapTuple    tuple;
-
-        MemoryContextCopyAndSetIdentifier(rscxt,
-                                          RelationGetRelationName(relation));
+    catalog = table_open(PolicyRelationId, AccessShareLock);
 
-        rsdesc = MemoryContextAllocZero(rscxt, sizeof(RowSecurityDesc));
-        rsdesc->rscxt = rscxt;
+    ScanKeyInit(&skey,
+                Anum_pg_policy_polrelid,
+                BTEqualStrategyNumber, F_OIDEQ,
+                ObjectIdGetDatum(RelationGetRelid(relation)));
 
-        catalog = table_open(PolicyRelationId, AccessShareLock);
+    sscan = systable_beginscan(catalog, PolicyPolrelidPolnameIndexId, true,
+                               NULL, 1, &skey);
 
-        ScanKeyInit(&skey,
-                    Anum_pg_policy_polrelid,
-                    BTEqualStrategyNumber, F_OIDEQ,
-                    ObjectIdGetDatum(RelationGetRelid(relation)));
+    while (HeapTupleIsValid(tuple = systable_getnext(sscan)))
+    {
+        Form_pg_policy policy_form = (Form_pg_policy) GETSTRUCT(tuple);
+        RowSecurityPolicy *policy;
+        Datum        datum;
+        bool        isnull;
+        char       *str_value;
 
-        sscan = systable_beginscan(catalog, PolicyPolrelidPolnameIndexId, true,
-                                   NULL, 1, &skey);
+        policy = MemoryContextAllocZero(rscxt, sizeof(RowSecurityPolicy));
 
         /*
-         * Loop through the row level security policies for this relation, if
-         * any.
+         * Note: we must be sure that pass-by-reference data gets copied into
+         * rscxt.  We avoid making that context current over wider spans than
+         * we have to, though.
          */
-        while (HeapTupleIsValid(tuple = systable_getnext(sscan)))
-        {
-            Datum        value_datum;
-            char        cmd_value;
-            bool        permissive_value;
-            Datum        roles_datum;
-            char       *qual_value;
-            Expr       *qual_expr;
-            char       *with_check_value;
-            Expr       *with_check_qual;
-            char       *policy_name_value;
-            bool        isnull;
-            RowSecurityPolicy *policy;
-
-            /*
-             * Note: all the pass-by-reference data we collect here is either
-             * still stored in the tuple, or constructed in the caller's
-             * short-lived memory context.  We must copy it into rscxt
-             * explicitly below.
-             */
-
-            /* Get policy command */
-            value_datum = heap_getattr(tuple, Anum_pg_policy_polcmd,
-                                       RelationGetDescr(catalog), &isnull);
-            Assert(!isnull);
-            cmd_value = DatumGetChar(value_datum);
-
-            /* Get policy permissive or restrictive */
-            value_datum = heap_getattr(tuple, Anum_pg_policy_polpermissive,
-                                       RelationGetDescr(catalog), &isnull);
-            Assert(!isnull);
-            permissive_value = DatumGetBool(value_datum);
-
-            /* Get policy name */
-            value_datum = heap_getattr(tuple, Anum_pg_policy_polname,
-                                       RelationGetDescr(catalog), &isnull);
-            Assert(!isnull);
-            policy_name_value = NameStr(*(DatumGetName(value_datum)));
-
-            /* Get policy roles */
-            roles_datum = heap_getattr(tuple, Anum_pg_policy_polroles,
-                                       RelationGetDescr(catalog), &isnull);
-            /* shouldn't be null, but initdb doesn't mark it so, so check */
-            if (isnull)
-                elog(ERROR, "unexpected null value in pg_policy.polroles");
-
-            /* Get policy qual */
-            value_datum = heap_getattr(tuple, Anum_pg_policy_polqual,
-                                       RelationGetDescr(catalog), &isnull);
-            if (!isnull)
-            {
-                qual_value = TextDatumGetCString(value_datum);
-                qual_expr = (Expr *) stringToNode(qual_value);
-            }
-            else
-                qual_expr = NULL;
 
-            /* Get WITH CHECK qual */
-            value_datum = heap_getattr(tuple, Anum_pg_policy_polwithcheck,
-                                       RelationGetDescr(catalog), &isnull);
-            if (!isnull)
-            {
-                with_check_value = TextDatumGetCString(value_datum);
-                with_check_qual = (Expr *) stringToNode(with_check_value);
-            }
-            else
-                with_check_qual = NULL;
+        /* Get policy command */
+        policy->polcmd = policy_form->polcmd;
 
-            /* Now copy everything into the cache context */
-            MemoryContextSwitchTo(rscxt);
+        /* Get policy, permissive or restrictive */
+        policy->permissive = policy_form->polpermissive;
 
-            policy = palloc0(sizeof(RowSecurityPolicy));
-            policy->policy_name = pstrdup(policy_name_value);
-            policy->polcmd = cmd_value;
-            policy->permissive = permissive_value;
-            policy->roles = DatumGetArrayTypePCopy(roles_datum);
-            policy->qual = copyObject(qual_expr);
-            policy->with_check_qual = copyObject(with_check_qual);
-            policy->hassublinks = checkExprHasSubLink((Node *) qual_expr) ||
-                checkExprHasSubLink((Node *) with_check_qual);
+        /* Get policy name */
+        policy->policy_name =
+            MemoryContextStrdup(rscxt, NameStr(policy_form->polname));
 
-            rsdesc->policies = lcons(policy, rsdesc->policies);
+        /* Get policy roles */
+        datum = heap_getattr(tuple, Anum_pg_policy_polroles,
+                             RelationGetDescr(catalog), &isnull);
+        /* shouldn't be null, but let's check for luck */
+        if (isnull)
+            elog(ERROR, "unexpected null value in pg_policy.polroles");
+        MemoryContextSwitchTo(rscxt);
+        policy->roles = DatumGetArrayTypePCopy(datum);
+        MemoryContextSwitchTo(oldcxt);
 
+        /* Get policy qual */
+        datum = heap_getattr(tuple, Anum_pg_policy_polqual,
+                             RelationGetDescr(catalog), &isnull);
+        if (!isnull)
+        {
+            str_value = TextDatumGetCString(datum);
+            MemoryContextSwitchTo(rscxt);
+            policy->qual = (Expr *) stringToNode(str_value);
             MemoryContextSwitchTo(oldcxt);
+            pfree(str_value);
+        }
+        else
+            policy->qual = NULL;
 
-            /* clean up some (not all) of the junk ... */
-            if (qual_expr != NULL)
-                pfree(qual_expr);
-            if (with_check_qual != NULL)
-                pfree(with_check_qual);
+        /* Get WITH CHECK qual */
+        datum = heap_getattr(tuple, Anum_pg_policy_polwithcheck,
+                             RelationGetDescr(catalog), &isnull);
+        if (!isnull)
+        {
+            str_value = TextDatumGetCString(datum);
+            MemoryContextSwitchTo(rscxt);
+            policy->with_check_qual = (Expr *) stringToNode(str_value);
+            MemoryContextSwitchTo(oldcxt);
+            pfree(str_value);
         }
+        else
+            policy->with_check_qual = NULL;
 
-        systable_endscan(sscan);
-        table_close(catalog, AccessShareLock);
-    }
-    PG_CATCH();
-    {
-        /* Delete rscxt, first making sure it isn't active */
+        /* We want to cache whether there are SubLinks in these expressions */
+        policy->hassublinks = checkExprHasSubLink((Node *) policy->qual) ||
+            checkExprHasSubLink((Node *) policy->with_check_qual);
+
+        /*
+         * Add this object to list.  For historical reasons, the list is built
+         * in reverse order.
+         */
+        MemoryContextSwitchTo(rscxt);
+        rsdesc->policies = lcons(policy, rsdesc->policies);
         MemoryContextSwitchTo(oldcxt);
-        MemoryContextDelete(rscxt);
-        PG_RE_THROW();
     }
-    PG_END_TRY();
 
-    /* Success --- attach the policy descriptor to the relcache entry */
+    systable_endscan(sscan);
+    table_close(catalog, AccessShareLock);
+
+    /*
+     * Success.  Reparent the descriptor's memory context under
+     * CacheMemoryContext so that it will live indefinitely, then attach the
+     * policy descriptor to the relcache entry.
+     */
+    MemoryContextSetParent(rscxt, CacheMemoryContext);
+
     relation->rd_rsdesc = rsdesc;
 }

On 25.09.2020 21:07, Tom Lane wrote:
> Konstantin Knizhnik <k.knizhnik@postgrespro.ru> writes:
>> On 25.09.2020 16:48, Tom Lane wrote:
>>> Hm ... this smells a whole lot like the issue we fixed recently for
>>> partition expressions, namely: what the devil are we doing reloading
>>> these expressions during RelationClearRelation?  We should delay
>>> populating that cache until the value is requested.
>> Sorry, this stack trace was obtained at 9.6 version of Postgres which
>> the customer is using.
>> So may be the problem with cache invalidation is already fixed.
> Not sure if we've changed anything much in that area.  I've occasionally
> wondered if we should not increase the size of the sinval message queue,
> but it hasn't happened yet.
>
> Anyway, looking more closely at RelationBuildRowSecurity, it doesn't
> seem to be doing anything more dangerous than RelationBuildRuleLock
> is; it's definitely not risking recursion, as the partition stuff did.
> So I take back the idea that we need to postpone it till the data is
> referenced.  However, it is leaking memory.
>
> Attached is a proposed rewrite that borrows some ideas from
> RelationBuildRuleLock, and also makes use of MemoryContextSetParent
> so we don't need a PG_TRY block.  In a very simple test
> (using regress_rls_schema.rls_tbl from the regression database),
> this doesn't seem to leak any memory in the caller context,
> and it also doesn't make the rscxt any bigger than it was before.
>
>> But at least calling pfree for a tree is very strange idea: we should
>> not do it all or allocate tree in separate memory context.
> Yeah, that's pretty useless.  But also I think the string forms of
> the expressions are leaking as much memory as the tree forms.
>
> This is against HEAD but I think it wouldn't be hard to back-patch.
> Are you in a position to see if it fixes the problem in your customer's
> environment?

Thank you.
Your patch fixes the problem.

Konstantin Knizhnik <k.knizhnik@postgrespro.ru> writes:
> On 25.09.2020 21:07, Tom Lane wrote:
>> This is against HEAD but I think it wouldn't be hard to back-patch.
>> Are you in a position to see if it fixes the problem in your customer's
>> environment?

> Thank you.
> Your patch fixes the problem.

Thanks for testing!  I'll try to get this committed later today.

            regards, tom lane

On 26.09.2020 20:05, Tom Lane wrote:
> Konstantin Knizhnik <k.knizhnik@postgrespro.ru> writes:
>> On 25.09.2020 21:07, Tom Lane wrote:
>>> This is against HEAD but I think it wouldn't be hard to back-patch.
>>> Are you in a position to see if it fixes the problem in your customer's
>>> environment?
>> Thank you.
>> Your patch fixes the problem.
> Thanks for testing!  I'll try to get this committed later today.
>
>             regards, tom lane

Just want to convey gratitude from the customer: after apply your patch 
memory consumption is decreased 3 times which significantly improves 
performance of their system.

-- 
Konstantin Knizhnik
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company