> -----Original Messages-----
> From: "Kevin Grittner" <kgrittn@gmail.com>
> > I tried 30 cores. But the CPU utilization is about 45%~70%.
> > How can we distinguish where the problem is? Is disk I/O or Lock?
>
> A simple way is to run `vmstat 1` for a bit during the test. Can
> you post a portion of the output of that here? If you can configure
> the WAL directory to a separate mount point (e.g., use the --waldir
> option of initdb), a snippet of `iostat 1` output would be even
> better.
"vmstat 1" output is as follow. Because I used only 30 cores (1/4 of all), cpu user time should be about 12*4 = 48.
There seems to be no process blocked by IO.
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
r b swpd free buff cache si so bi bo in cs us sy id wa st
28 0 0 981177024 315036 70843760 0 0 0 9 0 0 1 0 99 0 0
21 1 0 981178176 315036 70843784 0 0 0 0 25482 329020 12 3 85 0 0
18 1 0 981179200 315036 70843792 0 0 0 0 26569 323596 12 3 85 0 0
17 0 0 981175424 315036 70843808 0 0 0 0 25374 322992 12 4 85 0 0
12 0 0 981174208 315036 70843824 0 0 0 0 24775 321577 12 3 85 0 0
8 0 0 981179328 315036 70845336 0 0 0 0 13115 199020 6 2 92 0 0
13 0 0 981179200 315036 70845792 0 0 0 0 22893 301373 11 3 87 0 0
11 0 0 981179712 315036 70845808 0 0 0 0 26933 325728 12 4 84 0 0
30 0 0 981178304 315036 70845824 0 0 0 0 23691 315821 11 4 85 0 0
12 1 0 981177600 315036 70845832 0 0 0 0 29485 320166 12 4 84 0 0
32 0 0 981180032 315036 70845848 0 0 0 0 25946 316724 12 4 84 0 0
21 0 0 981176384 315036 70845864 0 0 0 0 24227 321938 12 4 84 0 0
21 0 0 981178880 315036 70845880 0 0 0 0 25174 326943 13 4 83 0 0
I used ramdisk to speedup the disk IO. Therefore, iostat can not give useful information.
> I think the best thing may be if you can generate a CPU flame graph
> of the worst case you can make for these lists:
> http://www.brendangregg.com/flamegraphs.html IMO, such a graph
> highlights the nature of the problem better than anything else.
>
The flame graph is attached. I use 'perf' to generate the flame graph. Only the CPUs running PG server are profiled.
I'm not familiar with other part of PG. Can you find anything unusual in the graph?
--
Mengxing Liu
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
