Re: 8xIntel S3500 SSD in RAID10 on Dell H710p

Поиск
Список
Период
Сортировка
От Graeme B. Bell
Тема Re: 8xIntel S3500 SSD in RAID10 on Dell H710p
Дата
Msg-id 93D2D314-57AE-403C-A939-F7E282E81956@skogoglandskap.no
обсуждение исходный текст
Ответ на 8xIntel S3500 SSD in RAID10 on Dell H710p  (Strahinja Kustudić <strahinjak@nordeus.com>)
Ответы Re: 8xIntel S3500 SSD in RAID10 on Dell H710p  (Strahinja Kustudić <strahinjak@nordeus.com>)
Список pgsql-performance
> 
> I have a beast of a Dell server with the following specifications:
>     • 4x Xeon E5-4657LV2 (48 cores total)
>     • 196GB RAM
>     • 2x SCSI 900GB in RAID1 (for the OS)
>     • 8x Intel S3500 SSD 240GB in RAID10
>     • H710p RAID controller, 1GB cache
> Centos 6.6, RAID10 SSDs uses XFS (mkfs.xfs -i size=512 /dev/sdb).

Things to check

- disk cache settings (EnDskCache - for SSD should be on or you're going to lose 90% of your performance)

- OS settings e.g. 

echo noop > /sys/block/sda/queue/scheduler
echo 975 > /sys/block/sda/queue/nr_requests
blockdev --setra 16384 /dev/sdb

- OS kernel version 

We use H710Ps with SSDs as well, and these settings make a measurable difference to our performance here (though we
measuremore than just pgbench since it's a poor proxy for our use cases).
 

Also

- SSDs - is the filesystem aligned and block size chosen correctly (you don't want to be forced to read 2 blocks of SSD
toget every data block)? RAID stripe size? May make a small difference. 
 

- are the SSDs all sitting on different SATA channels? You don't want them to be forced to share one channel's worth of
bandwidth.The H710P has 8 SATA channels I think (?) and you mention 10 devices above. 
 

Graeme Bell.

On 10 Dec 2014, at 00:28, Strahinja Kustudić <strahinjak@nordeus.com> wrote:

> I have a beast of a Dell server with the following specifications:
>     • 4x Xeon E5-4657LV2 (48 cores total)
>     • 196GB RAM
>     • 2x SCSI 900GB in RAID1 (for the OS)
>     • 8x Intel S3500 SSD 240GB in RAID10
>     • H710p RAID controller, 1GB cache
> Centos 6.6, RAID10 SSDs uses XFS (mkfs.xfs -i size=512 /dev/sdb).
> 
> Here are some relevant postgresql.conf settings:
> shared_buffers = 8GB
> work_mem = 64MB
> maintenance_work_mem = 1GB
> synchronous_commit = off
> checkpoint_segments = 256
> checkpoint_timeout = 10min
> checkpoint_completion_target = 0.9
> seq_page_cost = 1.0
> effective_cache_size = 100GB
> 
> I ran some "fast" pgbench tests with 4, 6 and 8 drives in RAID10 and here are the results:
> 
> time /usr/pgsql-9.1/bin/pgbench -U postgres -i -s 12000 pgbench # 292GB DB
> 
> 4 drives    6 drives    8 drives
> 105 min    98 min    94 min
> 
> /usr/pgsql-9.1/bin/pgbench -U postgres -c 96 -T 600 -N pgbench   # Write test
> 
> 4 drives    6 drives    8 drives
> 6567    7427    8073
> 
> /usr/pgsql-9.1/bin/pgbench -U postgres -c 96 -T 600 pgbench  # Read/Write test
> 
> 4 drives    6 drives    8 drives
> 3651    5474    7203
> 
> /usr/pgsql-9.1/bin/pgbench -U postgres -c 96 -T 600 -S pgbench  # Read test
> 
> 4 drives    6 drives    8 drives
> 17628    25482    28698
> 
> 
> A few notes:
>     • I ran these tests only once, so take these number with reserve. I didn't have the time to run them more times,
becauseI had to test how the server works with our app and it takes a considerable amount of time to run them all.
 
>     • I wanted to use a bigger scale factor, but there is a bug in pgbench with big scale factors.
>     • Postgres 9.1 was chosen, since the app which will run on this server uses 9.1.
>     • These tests are with the H710p controller set to write-back (WB) and with adaptive read ahead (ADRA). I ran a
fewtests with write-through (WT) and no read ahead (NORA), but the results were worse.
 
>     • All tests were run using 96 clients as recommended on the pgbench wiki page, but I'm sure I would get better
resultsif I used 48 clients (1 for each core), which I tried with the R/W test and got 7986 on 8 drives, which is
almost800TPS better than with 96 clients.
 
> 
> Since our app is tied to the Postgres performance a lot, I'm currently trying to optimize it. Do you have any
suggestionswhat Postgres/system settings I could try to tweak to increase performance? I have a feeling I could get
moreperformance out of this system.
 
> 
> 
> Regards,
> Strahinja


В списке pgsql-performance по дате отправления:

Предыдущее
От: Ruben Domingo Gaspar Aparicio
Дата:
Сообщение: Re: Postgres slave not catching up (on 9.2)
Следующее
От: "Graeme B. Bell"
Дата:
Сообщение: Re: Tuning the configuration