Re: Reliability with RAID 10 SSD and Streaming Replication

On 5/22/13 12:56 PM, Shaun Thomas wrote:
> Well, you may not be able to make that claim, but I can. While we don't
> use Intel SSDs, our first-gen FusinoIO cards can deliver about 20k
> PostgreSQL TPS of our real-world data right off the device before
> caching effects start boosting the numbers.

I've seen FusionIO hit that 20K commit number, as well as hitting 75K 
IOPS on random reads (600MB/s).  They are roughly 5 to 10X faster than 
the Intel 320/710 drives.  There's a corresponding price hit though, and 
having to provision PCI-E cards is a pain in some systems.

A claim that a FusionIO drive in particular is capable of 100X the 
performance of a spinning drive, that I wouldn't dispute.  I even made 
that claim myself with some benchmark numbers to back it up: 
http://www.fusionio.com/blog/fusion-io-boosts-postgresql-performance/ 
That's not just a generic SSD anymore though.

> An 8-drive 15k RPM RAID-10 gave us about 1800 TPS back when we switched
> to FusionIO about two years ago. So, while Intel drives themselves may
> not be able to hit sustained 100x speeds over spindles, it's pretty
> clear that that's a firmware or implementation limitation.

1800 TPS to 20K TPS is just over a 10X speedup.

As for Intel vs. FusionIO, rather than implementation quality it's more 
what architecture you're willing to pay for.  If you test a few models 
across Intel's product line, you can see there's a rough size vs. speed 
correlation.  The larger units have more channels of flash going at the 
same time.  FusionIO has architected such that there is a wide write 
path even on their smallest cards.  That 75K IOPS number I got even out 
of their little 80GB card.  (since dropped from the product line)

I can buy a good number of Intel DC S3700 drives for what a FusionIO 
card costs though.

> The main "issue" is that the sustained sequence scan speeds are
> generally less than an order of magnitude faster than drives. So as soon
> as you hit something that isn't limited by random IOPS, spindles get a
> chance to catch up.

I have some moderately fast SSD based transactional systems that are 
still using traditional drives with battery-backed cache for the 
sequential writes of the WAL volume, where the data volume is on Intel 
710 disks.  WAL writes really burn through flash cells, too, so keeping 
them on traditional drives can be cost effective in a few ways.  That 
approach is lucky to hit 10K TPS though, so it can't compete against 
what a PCI-E card like the FusionIO drives are capable of.

-- 
Greg Smith   2ndQuadrant US    greg@2ndQuadrant.com   Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com