Re: Intel 710 pgbench write latencies

---

From: Yeb Havinga <yebhavinga@gmail.com>
To: "pgsql-performance@postgresql.org" <pgsql-performance@postgresql.org>
Sent: Wednesday, November 2, 2011 9:05 AM
Subject: [PERFORM] Intel 710 pgbench write latencies

Hello list,

A OCZ Vertex 2 PRO and Intel 710 SSD, both 100GB, in a software raid 1 setup. I was pretty convinced this was the perfect solution to run PostgreSQL on SSDs without a IO controller with BBU. No worries for strange firmware bugs because of two different drives, good write endurance of the 710. Access to the smart attributes. Complete control over the disks: nothing hidden by a hardware raid IO layer.

Then I did a pgbench test:
- bigger than RAM test (~30GB database with 24GB ram)
- and during that test I removed the Intel 710.
- during the test I removed the 710 and 10 minutes later inserted it again and added it to the array.

The pgbench transaction latency graph is here: http://imgur.com/JSdQd

With only the OCZ, latencies are acceptable but with two drives, there are latencies up to 3 seconds! (and 11 seconds at disk remove time) Is this due to software raid, or is it the Intel 710? To figure that out I repeated the test, but now removing the OCZ, latency graph at: http://imgur.com/DQa59 (The 12 seconds maximum was at disk remove time.)

So the Intel 710 kind of sucks latency wise. Is it because it is also heavily reading, and maybe WAL should not be put on it?

I did another test, same as before but
* with 5GB database completely fitting in RAM (24GB)
* put WAL on a ramdisk
* started on the mirror
* during the test mdadm --fail on the Intel SSD

Latency graph is at: http://imgur.com/dY0Rk

So still: with Intel 710 participating in writes (beginning of graph), some latencies are over 2 seconds, with only the OCZ, max write latencies are near 300ms.

I'm now contemplating not using the 710 at all. Why should I not buy two 6Gbps SSDs without supercap (e.g. Intel 510 and OCZ Vertex 3 Max IOPS) with a IO controller+BBU?

Benefits: should be faster for all kinds of reads and writes.
Concerns: TRIM becomes impossible (which was already impossible with md raid1, lvm / dm based mirroring could work) but is TRIM important for a PostgreSQL io load, without e.g. routine TRUNCATES? Also the write endurance of these drives is probably a lot less than previous setup.

Thoughts, ideas are highly appreciated!
-- Yeb

PS:
I checked for proper alignment of partitions as well as md's data offsett, all was well.
Ext4 filesystem mounted with barrier=0
/proc/sys/vm/dirty_background_bytes set to 178500000



-- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance