Re: [NOVICE] - SAN/NAS/DAS - Need advises

Not sure if this will make it as I'm not subscribed.  If it does, please
CC me on any replies.

> Are you really comparing equal systems? "8gbit Fibre Channel" means a
> single Fibre Channel shared by 42 disks, whereas "3GBit DAS SAS"
> means 42 > > 3gbit channels running in parallel.

Saw this on osdir while Googling and I thought I'd try to quash
misinformation.  The above statement is factually incorrect on many
levels.  Let me explain:

The current, 2010, Nexsan SASBeast carries 42x15k SAS drives and has
_FOUR_ 8G FC ports, 2 per controller, for a total of 8.5GB/s
bidirectional aggregate raw FC bandwidth, not the single port 2.125GB/s
inferred by the OP.  42 dual ported 3Gb/s SAS drives have a
bidirectional aggregate maximum raw bandwidth of 25.2 GB/s, though
current SAS drives only hit about 150MB/s while streaming large blocks,
or half the SAS wire rate.  The ratio of raw max disk to raw max SAN
bandwidth with the Nexsan SASBeast is 2.96:1.

By the same token, if I read the MDS600 PDF correctly, with a dual
domain setup using dual ported SAS drives, a total of eight SFF8088
connectors on 4 I/o modules provide 32 SAS2 links for a maximum of
19.2GB/s bidirectional aggregate raw bandwidth.   70 dual ported 3Gb/s
SAS drives have a bidirectional aggregate maximum raw bandwidth of
42GB/s.  The ratio of raw max disk to raw max SAS2 wire bandwidth to the
DAS host is 2.19:1.

Note that both designs are using SAS Expanders to collate multiple
drives into a single upstream SAS channel, in the case of Nexsan, this
is to the SAN controllers, and in the case of HP it is to the host, or
possibly an SAS switch.

Noting that actual drive performance is limited to about 150MB/s, the 42
drives of the Nexsan can spin 6.3GB/s aggregate throughput, which is
well below the 8.5GB/s FC bandwidth of the SASBeast.  The 70 drives of
the MDS600 can spin 10.5GB/s aggregate throughput, which is well below
the max host bandwidth of 19.2GB/s.

In both cases, total performance is limited by the RAID controllers, not
the chassis backplane wiring or drive bandwidth.  In the case of the
SASBeast the relatively low RAID controller performance of this
inexpensive FC SAN array limits maximum sustained throughput to
approximately 1.2GB/s, depending on the RAID level used and the number
of spindles per array, even in the presence of 4GB of cache (2GB per
controller).  Sustained 1.2GB/s is more than plenty of disk bandwidth
for many, probably most, production applications.

The typical configuration of the MDS600 is to connect all 70 drives
through 8 SAS2 channels to an SAS2 switch, with a P700m RAID card with
256 or 512MB cache in each server attached to the MDS600.  The downside
to this JBOD SAS setup is that storage assignment to each server is at
the drive level.

So, if you have 70 disks and 7 servers, and you want an equal amount of
storage and performance per server, the maximum array you can create per
server uses only 10 disks.  If you want to do RAID6 with a spare for
each server, you've lost 3 disks per server to redundancy, or 21 your
seventy disks--almost 1/3rd of your disk capacity is lost to redundancy.
 Also, your stripe size is limited to 9 spindles (remember, 1 spare per
array), so each server will get a maximum of 7 spindles of post RAID6
parity stripe performance with a relatively low end RAID controller, for
somewhere around 300-400MB/s.

Now, with the SASBeast, as it's fiber channel, you can connect a huge
number of hosts.  The maximum number of LUNs you can export is 254, so
you could potentially assign LUNs to 254 servers on your SAN.  If you
want maximum performance and redundancy, you simply assign 2 disks as
spares, and dump the other 40 into a RAID6 array.  You can now carve up
this 40 drive array into say, 38 LUNs, assigning each to a different
server.  The LUNs are logical, and each is spread over the entire 40
drive array.  Each server will get all 38 spindles worth of stripe
performance because the RAID6 stripe is over 40 disks, not 9 as in the
MDS600 case.

There is a reason even inexpensive FC SAN arrays cost more than JBOD
storage systems.  Even though they may have less theoretical bandwidth,
in deployment, they have equal or better performance for many workloads,
and far far greater flexibility.  With an SASBeast (or any Nexsan array)
I can reassign, say, a 3TB LUN from a downed production database server
with a dead motherboard to standby server with a few mouse clicks, power
on the spare server, and have it boot directly from the SAN LUN as the
previous server did.  My production db server is back up in under 2
minutes--from the time I'm aware of the dead server that is.  Try doing
that with a DAS JBOD MDS600. :)  Well, ok, if you're running a VMWare
ESX cluster with multiple hosts connected to the MDS600 you could do it
just as fast.  Same is true for the SASBeast, but in that case, I could
have 254 ESX hosts in my cluster. :)

--
Stan