Relation extension lock bottleneck

Поиск
Список
Период
Сортировка
От Konstantin Knizhnik
Тема Relation extension lock bottleneck
Дата
Msg-id bae102f3-26bb-065b-0b37-32134db32427@postgrespro.ru
обсуждение исходный текст
Список pgsql-hackers
Hi hackers,

We have a customer which suffer from Postgres performance degradation 
when there are large number of connections performing inserts in the 
same table.
In 2016 Robert Haas has committed optimization of relation extension 
719c84c1:

Author: Robert Haas <rhaas@postgresql.org>
Date:   Fri Apr 8 02:04:46 2016 -0400

     Extend relations multiple blocks at a time to improve scalability.

     Contention on the relation extension lock can become quite fierce when
     multiple processes are inserting data into the same relation at the 
same
     time at a high rate.  Experimentation shows the extending the relation
     multiple blocks at a time improves scalability.

But this optimization is applied only for heap relations 
(RelationGetBufferForTuple).
And here most of backends are competing for index relation extension lock:

         /*
          * Extend the relation by one page.
          *
          * We have to use a lock to ensure no one else is extending the 
rel at
          * the same time, else we will both try to initialize the same new
          * page.  We can skip locking for new or temp relations, however,
          * since no one else could be accessing them.
          */
         needLock = !RELATION_IS_LOCAL(rel);

         if (needLock)
             LockRelationForExtension(rel, ExclusiveLock);


#0  0x00007ff1787065e3 in __epoll_wait_nocancel () from /lib64/libc.so.6
#1  0x000000000072e39e in WaitEventSetWaitBlock (nevents=1, 
occurred_events=0x7ffd03c0ddf0, cur_timeout=-1, set=0x2cb1838) at 
latch.c:1048
#2  WaitEventSetWait (set=set@entry=0x2cb1838, timeout=timeout@entry=-1, 
occurred_events=occurred_events@entry=0x7ffd03c0ddf0, 
nevents=nevents@entry=1, wait_event_info=wait_event_info@entry=50331649) a
t latch.c:1000
#3  0x000000000072e7fb in WaitLatchOrSocket (latch=0x2aec0cf5c844, 
wakeEvents=wakeEvents@entry=1, sock=sock@entry=-1, timeout=-1, 
timeout@entry=0, wait_event_info=50331649) at latch.c:385
#4  0x000000000072e8b0 in WaitLatch (latch=<optimized out>, 
wakeEvents=wakeEvents@entry=1, timeout=timeout@entry=0, 
wait_event_info=<optimized out>) at latch.c:339
#5  0x000000000073e2c6 in ProcSleep 
(locallock=locallock@entry=0x2ace708, 
lockMethodTable=lockMethodTable@entry=0x9cee80 <default_lockmethod>) at 
proc.c:1284
#6  0x0000000000738d92 in WaitOnLock 
(locallock=locallock@entry=0x2ace708, owner=owner@entry=0x28f2d10) at 
lock.c:1750
#7  0x000000000073a216 in LockAcquireExtended 
(locktag=locktag@entry=0x7ffd03c0e170, lockmode=lockmode@entry=7, 
sessionLock=sessionLock@entry=false, dontWait=dontWait@entry=false, 
reportMemoryError=rep
ortMemoryError@entry=true, locallockp=locallockp@entry=0x0) at lock.c:1032
#8  0x000000000073a8d4 in LockAcquire 
(locktag=locktag@entry=0x7ffd03c0e170, lockmode=lockmode@entry=7, 
sessionLock=sessionLock@entry=false, dontWait=dontWait@entry=false) at 
lock.c:695
#9  0x0000000000737c36 in LockRelationForExtension 
(relation=relation@entry=0x3089c30, lockmode=lockmode@entry=7) at lmgr.c:362
#10 0x00000000004d2209 in _bt_getbuf (rel=rel@entry=0x3089c30, 
blkno=blkno@entry=4294967295, access=access@entry=2) at nbtpage.c:829
#11 0x00000000004d013b in _bt_split (newitemonleft=true, 
newitem=0x2cb15b8, newitemsz=24, newitemoff=63, firstright=138, cbuf=0, 
buf=27480727, rel=0x3089c30) at nbtinsert.c:1156
#12 _bt_insertonpg (rel=rel@entry=0x3089c30, buf=buf@entry=27480727, 
cbuf=cbuf@entry=0, stack=stack@entry=0x2cb1758, 
itup=itup@entry=0x2cb15b8, newitemoff=63, 
split_only_page=split_only_page@entry=fals
e) at nbtinsert.c:909
#13 0x00000000004d1b1c in _bt_doinsert (rel=rel@entry=0x3089c30, 
itup=itup@entry=0x2cb15b8, 
checkUnique=checkUnique@entry=UNIQUE_CHECK_NO, 
heapRel=heapRel@entry=0x3088d70) at nbtinsert.c:306
#14 0x00000000004d4651 in btinsert (rel=0x3089c30, values=<optimized 
out>, isnull=<optimized out>, ht_ctid=0x2bd230c, heapRel=0x3088d70, 
checkUnique=UNIQUE_CHECK_NO, indexInfo=0x2caf828) at nbtree.c:20
5
#15 0x000000000060a41a in ExecInsertIndexTuples 
(slot=slot@entry=0x2c875f0, tupleid=tupleid@entry=0x2bd230c, 
estate=estate@entry=0x2c85dc0, noDupErr=noDupErr@entry=false, 
specConflict=specConflict@entr
y=0x0, arbiterIndexes=arbiterIndexes@entry=0x0) at execIndexing.c:386
#16 0x000000000062d132 in ExecInsert (mtstate=mtstate@entry=0x2c86110, 
slot=0x2c875f0, planSlot=planSlot@entry=0x2c875f0, 
estate=estate@entry=0x2c85dc0, canSetTag=<optimized out>) at 
nodeModifyTable.c:
535
#17 0x000000000062e1b9 in ExecModifyTable (pstate=0x2c86110) at 
nodeModifyTable.c:2159


I wonder if such optimization should also be used for index relations?
Can we just  make RelationAddExtraBlocks public (non static) and use it 
in B-Tree code in the same way as in hio.c?
Or it is better to provide some special function for extending arbitrary 
relation?

-- 
Konstantin Knizhnik
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company




В списке pgsql-hackers по дате отправления:

Предыдущее
От: Alvaro Herrera
Дата:
Сообщение: Re: Commit fest 2019-09
Следующее
От: ilmari@ilmari.org (Dagfinn Ilmari Mannsåker)
Дата:
Сообщение: Re: Proposal: Add more compile-time asserts to expose inconsistencies.