Re: Patch: fix lock contention for HASHHDR.mutex

Поиск
Список
Период
Сортировка
Искать
От
Aleksander Alekseev
Тема
Re: Patch: fix lock contention for HASHHDR.mutex
Дата
Msg-id
20160210112400.22c0b677@fujitsu
Ответ на
Список
Дерево обсуждения
Patch: fix lock contention for HASHHDR.mutex Aleksander Alekseev <a.alekseev@postgrespro.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Andres Freund <andres@anarazel.de>
Re: Patch: fix lock contention for HASHHDR.mutex Robert Haas <robertmhaas@gmail.com>
Re: Patch: fix lock contention for HASHHDR.mutex Andres Freund <andres@anarazel.de>
Re: Patch: fix lock contention for HASHHDR.mutex Aleksander Alekseev <a.alekseev@postgrespro.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Robert Haas <robertmhaas@gmail.com>
Re: Patch: fix lock contention for HASHHDR.mutex Aleksander Alekseev <a.alekseev@postgrespro.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Aleksander Alekseev <a.alekseev@postgrespro.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Andres Freund <andres@anarazel.de>
Re: Patch: fix lock contention for HASHHDR.mutex Aleksander Alekseev <a.alekseev@postgrespro.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Teodor Sigaev <teodor@sigaev.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Aleksander Alekseev <a.alekseev@postgrespro.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Robert Haas <robertmhaas@gmail.com>
Re: Patch: fix lock contention for HASHHDR.mutex Aleksander Alekseev <a.alekseev@postgrespro.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Robert Haas <robertmhaas@gmail.com>
Re: Patch: fix lock contention for HASHHDR.mutex Aleksander Alekseev <a.alekseev@postgrespro.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Aleksander Alekseev <a.alekseev@postgrespro.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Alvaro Herrera <alvherre@2ndquadrant.com>
Re: Patch: fix lock contention for HASHHDR.mutex Andres Freund <andres@anarazel.de>
Re: Patch: fix lock contention for HASHHDR.mutex Alvaro Herrera <alvherre@2ndquadrant.com>
Re: Patch: fix lock contention for HASHHDR.mutex Aleksander Alekseev <a.alekseev@postgrespro.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Oleg Bartunov <obartunov@gmail.com>
--enable-depend by default (was Re: Patch: fix lock contention for HASHHDR.mutex) Tom Lane <tgl@sss.pgh.pa.us>
Re: --enable-depend by default (was Re: Patch: fix lock contention for HASHHDR.mutex) Andres Freund <andres@anarazel.de>
Re: --enable-depend by default (was Re: Patch: fix lock contention for HASHHDR.mutex) Greg Stark <stark@mit.edu>
Re: --enable-depend by default (was Re: Patch: fix lock contention for HASHHDR.mutex) Tom Lane <tgl@sss.pgh.pa.us>
Re: --enable-depend by default (was Re: Patch: fix lock contention for HASHHDR.mutex) Andres Freund <andres@anarazel.de>
Re: Patch: fix lock contention for HASHHDR.mutex Fabrízio de Royes Mello <fabriziomello@gmail.com>
Re: Patch: fix lock contention for HASHHDR.mutex Aleksander Alekseev <a.alekseev@postgrespro.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Alvaro Herrera <alvherre@2ndquadrant.com>
Re: Patch: fix lock contention for HASHHDR.mutex Aleksander Alekseev <a.alekseev@postgrespro.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Dilip Kumar <dilipbalaut@gmail.com>
Re: Patch: fix lock contention for HASHHDR.mutex Aleksander Alekseev <a.alekseev@postgrespro.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Dilip Kumar <dilipbalaut@gmail.com>
Re: Patch: fix lock contention for HASHHDR.mutex Aleksander Alekseev <a.alekseev@postgrespro.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Dilip Kumar <dilipbalaut@gmail.com>
Re: Patch: fix lock contention for HASHHDR.mutex Andres Freund <andres@anarazel.de>
Re: Patch: fix lock contention for HASHHDR.mutex Alvaro Herrera <alvherre@2ndquadrant.com>
Re: Patch: fix lock contention for HASHHDR.mutex Anastasia Lubennikova <a.lubennikova@postgrespro.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Aleksander Alekseev <a.alekseev@postgrespro.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Anastasia Lubennikova <a.lubennikova@postgrespro.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Aleksander Alekseev <a.alekseev@postgrespro.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Robert Haas <robertmhaas@gmail.com>
Re: Patch: fix lock contention for HASHHDR.mutex Aleksander Alekseev <a.alekseev@postgrespro.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Robert Haas <robertmhaas@gmail.com>
Re: Patch: fix lock contention for HASHHDR.mutex Alvaro Herrera <alvherre@2ndquadrant.com>
Re: Patch: fix lock contention for HASHHDR.mutex Aleksander Alekseev <a.alekseev@postgrespro.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Robert Haas <robertmhaas@gmail.com>
Re: Patch: fix lock contention for HASHHDR.mutex Aleksander Alekseev <a.alekseev@postgrespro.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Robert Haas <robertmhaas@gmail.com>
Re: Patch: fix lock contention for HASHHDR.mutex Robert Haas <robertmhaas@gmail.com>
Re: Patch: fix lock contention for HASHHDR.mutex Aleksander Alekseev <a.alekseev@postgrespro.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Amit Kapila <amit.kapila16@gmail.com>
Re: Patch: fix lock contention for HASHHDR.mutex Aleksander Alekseev <a.alekseev@postgrespro.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Robert Haas <robertmhaas@gmail.com>
Re: Patch: fix lock contention for HASHHDR.mutex Amit Kapila <amit.kapila16@gmail.com>
Re: Patch: fix lock contention for HASHHDR.mutex Robert Haas <robertmhaas@gmail.com>
Re: Patch: fix lock contention for HASHHDR.mutex Amit Kapila <amit.kapila16@gmail.com>
Re: Patch: fix lock contention for HASHHDR.mutex Robert Haas <robertmhaas@gmail.com>
Re: Patch: fix lock contention for HASHHDR.mutex Amit Kapila <amit.kapila16@gmail.com>
Re: Patch: fix lock contention for HASHHDR.mutex Aleksander Alekseev <a.alekseev@postgrespro.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Robert Haas <robertmhaas@gmail.com>
Re: Patch: fix lock contention for HASHHDR.mutex Aleksander Alekseev <a.alekseev@postgrespro.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Robert Haas <robertmhaas@gmail.com>
Re: Patch: fix lock contention for HASHHDR.mutex Aleksander Alekseev <a.alekseev@postgrespro.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Amit Kapila <amit.kapila16@gmail.com>
Re: Patch: fix lock contention for HASHHDR.mutex Aleksander Alekseev <a.alekseev@postgrespro.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Amit Kapila <amit.kapila16@gmail.com>
Re: Patch: fix lock contention for HASHHDR.mutex Robert Haas <robertmhaas@gmail.com>
Re: Patch: fix lock contention for HASHHDR.mutex Amit Kapila <amit.kapila16@gmail.com>
Re: Patch: fix lock contention for HASHHDR.mutex Aleksander Alekseev <a.alekseev@postgrespro.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Amit Kapila <amit.kapila16@gmail.com>
Re: Patch: fix lock contention for HASHHDR.mutex Aleksander Alekseev <a.alekseev@postgrespro.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Amit Kapila <amit.kapila16@gmail.com>
Re: Patch: fix lock contention for HASHHDR.mutex Tom Lane <tgl@sss.pgh.pa.us>
Re: Patch: fix lock contention for HASHHDR.mutex Aleksander Alekseev <a.alekseev@postgrespro.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Aleksander Alekseev <a.alekseev@postgrespro.ru>
Re: Patch: fix lock contention for HASHHDR.mutex Simon Riggs <simon@2ndQuadrant.com>
Re: Patch: fix lock contention for HASHHDR.mutex Aleksander Alekseev <a.alekseev@postgrespro.ru>
Hello, Robert

> Basically, the burden for you to impose a new coding rule on everybody
> who uses shared hash tables in the future is very high.

I fixed an issue you described. Number of spinlocks doesn't depend of
NUM_LOCK_PARTITIONS anymore and could be specified for each hash table
on a calling side.

I did a benchmark described in a first message of this thread again.
Currently I don't have access to the same 60-core server so I used more
common 12-core server (24 with HT). According to this benchmark TPS
increment depends on NUM_LOCK_PARTITIONS and default number of
spinlocks this way:

pgbench -f pgbench.sql -T 150 -P 1 -c 40 -j 12

 DMN | NLP = 16 | NLP = 32 | NLP = 64 | NLP = 128
-----|----------|----------|----------|----------
   8 |  +15.1%  |  +28.2%  |  +34.1%  |  +33.7%  
  16 |  +16.6%  |  +30.9%  |  +37.0%  |  +40.8%  
  32 |  +15.1%  |  +33.9%  |  +39.5%  |  +41.9%  
  64 |  +15.0%  |  +31.9%  |  +40.1%  |  +42.9%  
 128 |   +7.7%  |  +24.7%  |  +29.6%  |  +31.6%  

* NLP = NUM_LOCK_PARTITIONS
* DMN = DEFAULT_MUTEXES_NUM

I realize this benchmark doesn't represent any possible workload so for
attached patch I choose NUM_LOCK_PARTITIONS = DEFAULT_MUTEXES_NUM = 32.
It seems to be a reasonable compromise of a speedup according to
"synthetic and meaningless in practice" benchmark and number of used
locks which could mean quite a lot in practice. Still this values could
be easily changed in any moment.

Here are before/after benchmark results for this concrete patch.


BEFORE

pgbench (default):

tps = 1295.798531 (including connections establishing)
tps = 1295.858295 (excluding connections establishing)

pgbench -f pgbench.sql:

tps = 1020.072172 (including connections establishing)
tps = 1020.116888 (excluding connections establishing)


AFTER

pgbench (default):

tps = 1299.369807 (including connections establishing)
tps = 1299.429764 (excluding connections establishing)

pgbench -f pgbench.sql:

tps = 1365.749333 (including connections establishing)
tps = 1365.814384 (excluding connections establishing)


So as I understand this patch solves a lock contention problem and
doesn't make anything else worse.
В списке pgsql-hackers по дате отправления
От: Michael Paquier
Дата:
От: Kyotaro HORIGUCHI
Дата:
FAQ