Re: Reliability with RAID 10 SSD and Streaming Replication

On 5/22/13 9:30 AM, Merlin Moncure wrote:
> That's most certainly *not* the only gain to be had: random read rates
> of large databases (a very important metric for data analysis) can
> easily hit 20k tps.  So I'll stand by the figure.

They can easily hit that number.  Or they can do this:

Device:     r/s    w/s  rMB/s  wMB/s avgrq-sz avgqu-sz  await svctm  %util
sdd     2702.80  19.40  19.67   0.16    14.91   273.68  71.74  0.37 100.00
sdd     2707.60  13.00  19.53   0.10    14.78   276.61  90.34  0.37 100.00

That's an Intel 710 being crushed by a random read database server
workload, unable to deliver even 3000 IOPS / 20MB/s.  I have hours of
data like this from several servers.  Yes, the DC S3700 drives are at
least twice as fast on average, but I haven't had one for long enough to
see what its worst case really looks like yet.

Here's a mechanical drive hitting its limits on the same server as the
above:

Device:     r/s     w/s  rMB/s  wMB/s avgrq-sz avgqu-sz   await  svctm
%util
sdb      100.80  220.60   1.06   1.79    18.16   228.78  724.11   3.11
100.00
sdb      119.20  220.40   1.09   1.77    17.22   228.36  677.46   2.94
100.00

Giving around 3MB/s.  I am quite happy saying the SSD is delivering
about a single order of magnitude improvement, in both throughput and
latency.  But that's it, and a single order of magnitude improvement is
sometimes not good enough to solve all storage issues.

If all you care about is speed, the main situation where I've found
there to still be value in "tier 1 storage" are extremely write-heavy
workloads.  The best write numbers I've seen out of Postgres are still
going into a monster EMC unit, simply because the unit I was working
with had 16GB of durable cache.  Yes, that only supports burst speeds,
but 16GB absorbs a whole lot of writes before it fills.  Write
re-ordering and combining can accelerate traditional disk quite a bit
when it's across a really large horizon like that.

> Anyways, SSD installation in the post-capacitor era has been 100.0%
> correlated in my experience (admittedly, around a dozen or so systems)
> with removal of storage as the primary performance bottleneck, and
> I'll stand by that.

I wish it were that easy for everyone, but that's simply not true.  Are
there lots of systems where SSD makes storage look almost free it's so
fast?  Sure.  But presuming all systems will look like that is
optimistic, and it sets unreasonable expectations.

--
Greg Smith   2ndQuadrant US    greg@2ndQuadrant.com   Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com