Re: Speed up Clog Access by increasing CLOG buffers

Поиск
Список
Период
Сортировка
От Amit Kapila
Тема Re: Speed up Clog Access by increasing CLOG buffers
Дата
Msg-id CAA4eK1L4iV-2qe7AyMVsb+nz7SiX8JvCO+CqhXwaiXgm3CaBUw@mail.gmail.com
обсуждение исходный текст
Ответ на Re: Speed up Clog Access by increasing CLOG buffers  (Robert Haas <robertmhaas@gmail.com>)
Ответы Re: Speed up Clog Access by increasing CLOG buffers  (Amit Kapila <amit.kapila16@gmail.com>)
Список pgsql-hackers
Дерево обсуждения
On Tue, Feb 23, 2016 at 7:06 PM, Robert Haas <robertmhaas@gmail.com> wrote:
On Sun, Feb 21, 2016 at 7:45 PM, Amit Kapila <amit.kapila16@gmail.com> wrote:
I mean, my basic feeling is that I would not accept a 2-3% regression in the single client case to get a 10% speedup in the case where we have 128 clients.


When I tried by running the pgbench first with patch and then with Head, I see 1.2% performance increase with patch.  TPS with patch is 976 and with Head it is 964. For 3, 30 mins TPS data, refer "Patch – group_clog_update_v5" and before that "HEAD – Commit 481725c0" in perf_write_clogcontrollock_data_v6.ods attached with this mail.

Nonetheless, I have observed that below new check has been added by the patch which can effect single client performance.  So I have changed it such that new check is done only when we there is actually a need of group update which means when multiple clients tries to update clog at-a-time.

+       if (!InRecovery &&
+               all_trans_same_page &&
+               nsubxids < PGPROC_MAX_CACHED_SUBXIDS &&
+               !IsGXactActive())

 
I understand your point.  I think to verify whether it is run-to-run
variation or an actual regression, I will re-run these tests on single
client multiple times and post the result.

Perhaps you could also try it on a couple of different machines (e.g. MacBook Pro and a couple of different large servers).

 
Okay, I have tried latest patch (group_update_clog_v6.patch) on 2 different big servers and then on Mac-Pro. The detailed data for various runs can be found in attached document perf_write_clogcontrollock_data_v6.ods.  I have taken the performance data for higher client-counts with somewhat larger scale factor (1000) and data for median of same is as below: 

M/c configuration
-----------------------------
RAM - 500GB
8 sockets, 64 cores(Hyperthreaded128 threads total)

Non-default parameters
------------------------------------
max_connections = 1000
shared_buffers=32GB
min_wal_size=10GB
max_wal_size=15GB
checkpoint_timeout    =35min
maintenance_work_mem = 1GB
checkpoint_completion_target = 0.9
wal_buffers = 256MB


Client_Count/Patch_ver1864128256
HEAD8715090177601761613907
PATCH9005110183312027719263


Here, we can see that there is a gain of ~15% to ~38% at higher client count.

The attached document (perf_write_clogcontrollock_data_v6.ods) contains data, mainly focussing on single client performance.  The data is for multiple runs on different machines, so I thought it is better to present in form of document rather than dumping everything in e-mail.  Do let me know if there is any confusion in understanding/interpreting the data.

Thanks to Dilip Kumar for helping me in conducting test of this patch on MacBook-Pro.

With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com
Вложения

В списке pgsql-hackers по дате отправления:

Предыдущее
От: Robert Haas
Дата:
Сообщение: Re: [COMMITTERS] pgsql: Respect TEMP_CONFIG when running contrib regression tests.
Следующее
От: Amit Kapila
Дата:
Сообщение: Re: Speed up Clog Access by increasing CLOG buffers