Обсуждение: New hardware thoughts

Hello all,

I am currently working out the best type of machine for a high volume
pgsql database that I going to need for a project. I will be
purchasing a new server specifically for the database, and it won't
be running any other applications. I will be using FreeBSD 6.1 Stable.

I think it may be beneficial if I give a brief overview of the types
of database access. There are several groups of tables and associated
accesses to them.

The first can be thought of as users details and configuration
tables. They will have low read and write access (say around 10 - 20
a min). SIzed at around 1/2 Million rows.

The second part is logging, this will be used occasionally for reads
when reports are run, but I will probably back that off to more
aggregated data tables, so can probably think of this as a write only
tables. Several table will each have around 200-300 inserts a second.
The can be archived on a regular basis to keep the size down, may be
once a day, or once a week. Not sure yet.

The third part will be transactional and will have around 50
transaction a second. A transaction is made up of a query followed by
an update, followed by approx 3 inserts. In addition some of these
tables will be read out of the transactions at approx once per second.

There will be around 50 simultaneous connections.

I hope that overview is a) enough and b) useful background to this
discussion.

I have some thoughts but I need them validating / discussing. If I
had the money I could buy the hardware and sped time testing
different options, thing is I need to get this pretty much right on
the hardware front first time. I'll almost certainly be buying Dell
kit, but could go for HP as an alternative.

Processor : I understand that pgsql is not CPU intensive, but that
each connection uses its own process. The HW has an option of upto 4
dual core xeon processors. My thoughts would be that more lower spec
processors would be better than fewer higher spec ones. But the
question is 4 (8 cores) wasted because there will be so much blocking
on I/O. Is 2 (4 cores) processors enough. I was thinking 2 x 2.6G
dual core Xeons would be enough.

Memory : I know this is very important for pgsql, and the more you
have the more of the tables can reside in memory. I was thinking of
around 8 - 12G, but the machine can hold a lot more. Thing is memory
is still quite expensive, and so I don't to over spec it if its not
going to get used.

Disk : Ok so this is the main bottleneck of the system. And the thing
I know least about, so need the most help with. I understand you get
good improvements if you keep the transaction log on a different disk
from the database, and that raid 5 is not as good as people think
unless you have lots of disks.

My option in disks is either 5 x 15K rpm disks or 8 x 10K rpm disks
(all SAS), or if I pick a different server I can have 6 x 15K rpm or
8 x 10K rpm (again SAS). In each case controlled by a PERC 5/i (which
I think is an LSI Mega Raid SAS 8408E card).

So the question here is will more disks at a slower speed be better
than fewer disks as a higher speed?

Assuming I was going to have a mirrored pair for the O/S and
transaction logs that would leave me with 3 or 4 15K rpm for the
database, 3 would mean raid 5 (not great at 3 disks), 4 would give me
raid 10 option if I wanted it.  Or I could have raid 5 across all 5/6
disks and not separate the transaction and database onto different
disks. Better performance from raid 5 with more disks, but does
having the transaction logs and database on the same disks
counteract / worsen the performance?

If I had the 8 10K disks, I could have 2 as a mirrored pair for O/S
Transaction, and still have 6 for raid 5. But the disks are slower.

Anybody have any good thoughts on my disk predicament, and which
options will serve me better.

Your thoughts are much appreciated.

Regards

Ben

Ben Suffolk wrote:
> Hello all,
>
> I am currently working out the best type of machine for a high volume
> pgsql database that I going to need for a project. I will be purchasing
> a new server specifically for the database, and it won't be running any
> other applications. I will be using FreeBSD 6.1 Stable.
>
> I think it may be beneficial if I give a brief overview of the types of
> database access. There are several groups of tables and associated
> accesses to them.
>
> The first can be thought of as users details and configuration tables.
> They will have low read and write access (say around 10 - 20 a min).
> SIzed at around 1/2 Million rows.
>
> The second part is logging, this will be used occasionally for reads
> when reports are run, but I will probably back that off to more
> aggregated data tables, so can probably think of this as a write only
> tables. Several table will each have around 200-300 inserts a second.
> The can be archived on a regular basis to keep the size down, may be
> once a day, or once a week. Not sure yet.
>
 > The third part will be transactional and will have around 50
 > transaction a second. A transaction is made up of a query followed by
 > an update, followed by approx 3 inserts. In addition some of these
 > tables will be read out of the transactions at approx once per second.
 >
> There will be around 50 simultaneous connections.
 >
 > I hope that overview is a) enough and b) useful background to this
 > discussion.

Sounds like you have a very good idea of what to expect. Are these solid
stats or certain estimates? Estimates can vary when it comes time to start.

> Processor : I understand that pgsql is not CPU intensive, but that each
> connection uses its own process. The HW has an option of upto 4 dual
> core xeon processors. My thoughts would be that more lower spec
> processors would be better than fewer higher spec ones. But the question
> is 4 (8 cores) wasted because there will be so much blocking on I/O. Is
> 2 (4 cores) processors enough. I was thinking 2 x 2.6G dual core Xeons
> would be enough.

I would think 2 will cope with what you describe but what about in 12
months time? Can you be sure your needs won't increase? And will the
cost of 4 CPU's cut your other options? If all 50 users may be running
the 3rd part at the same time (or is that your 50 trans. a second?) then
I'd consider the 4.

> Memory : I know this is very important for pgsql, and the more you have
> the more of the tables can reside in memory. I was thinking of around 8
> - 12G, but the machine can hold a lot more. Thing is memory is still
> quite expensive, and so I don't to over spec it if its not going to get
> used.

8GB is a good starting point for a busy server but a few hundred $ on
the extra ram can make more difference than extra disks (more for the
reading part than writing).

What you describe plans several times 300 inserts to logging plus 150
inserts and 50 updates and 1 read a second plus occasional reads to the
logging and user data.
Will it be raw data fed in and saved or will the server be calculating a
majority of the inserted data? If so go for the 4 cpu's.

Again allow room for expansion.

> Disk : Ok so this is the main bottleneck of the system. And the thing I
> know least about, so need the most help with. I understand you get good
> improvements if you keep the transaction log on a different disk from
> the database, and that raid 5 is not as good as people think unless you
> have lots of disks.
>
> My option in disks is either 5 x 15K rpm disks or 8 x 10K rpm disks (all
> SAS), or if I pick a different server I can have 6 x 15K rpm or 8 x 10K
> rpm (again SAS). In each case controlled by a PERC 5/i (which I think is
> an LSI Mega Raid SAS 8408E card).
>
> So the question here is will more disks at a slower speed be better than
> fewer disks as a higher speed?

Generally more disks at slower speed - 2 10K disks in raid 0 is faster
than 1 15K disk. More disks also allow more options.

Choosing the best RAID controller can make a lot of difference too.

> Assuming I was going to have a mirrored pair for the O/S and transaction
> logs that would leave me with 3 or 4 15K rpm for the database, 3 would
> mean raid 5 (not great at 3 disks), 4 would give me raid 10 option if I
> wanted it.  Or I could have raid 5 across all 5/6 disks and not separate
> the transaction and database onto different disks. Better performance
> from raid 5 with more disks, but does having the transaction logs and
> database on the same disks counteract / worsen the performance?
>
> If I had the 8 10K disks, I could have 2 as a mirrored pair for O/S
> Transaction, and still have 6 for raid 5. But the disks are slower.
>

I might consider RAID 5 with 8 disks but would lean more for 2 RAID 10
setups. This can give you the reliability and speed with system and xlog
on one and data on the other.

Sounds to me like you have it worked out even if you are a little
indecisive on a couple of finer points.


--

Shane Ambler
Postgres@007Marketing.com

Get Sheeky @ http://Sheeky.Biz

Ben,

On 20-Oct-06, at 3:49 AM, Ben Suffolk wrote:

> Hello all,
>
> I am currently working out the best type of machine for a high
> volume pgsql database that I going to need for a project. I will be
> purchasing a new server specifically for the database, and it won't
> be running any other applications. I will be using FreeBSD 6.1 Stable.
>
> I think it may be beneficial if I give a brief overview of the
> types of database access. There are several groups of tables and
> associated accesses to them.
>
> The first can be thought of as users details and configuration
> tables. They will have low read and write access (say around 10 -
> 20 a min). SIzed at around 1/2 Million rows.
>
> The second part is logging, this will be used occasionally for
> reads when reports are run, but I will probably back that off to
> more aggregated data tables, so can probably think of this as a
> write only tables. Several table will each have around 200-300
> inserts a second. The can be archived on a regular basis to keep
> the size down, may be once a day, or once a week. Not sure yet.
>
> The third part will be transactional and will have around 50
> transaction a second. A transaction is made up of a query followed
> by an update, followed by approx 3 inserts. In addition some of
> these tables will be read out of the transactions at approx once
> per second.
>
> There will be around 50 simultaneous connections.
>
> I hope that overview is a) enough and b) useful background to this
> discussion.
>
> I have some thoughts but I need them validating / discussing. If I
> had the money I could buy the hardware and sped time testing
> different options, thing is I need to get this pretty much right on
> the hardware front first time. I'll almost certainly be buying Dell
> kit, but could go for HP as an alternative.
>
> Processor : I understand that pgsql is not CPU intensive, but that
> each connection uses its own process. The HW has an option of upto
> 4 dual core xeon processors. My thoughts would be that more lower
> spec processors would be better than fewer higher spec ones. But
> the question is 4 (8 cores) wasted because there will be so much
> blocking on I/O. Is 2 (4 cores) processors enough. I was thinking 2
> x 2.6G dual core Xeons would be enough.
>
> Memory : I know this is very important for pgsql, and the more you
> have the more of the tables can reside in memory. I was thinking of
> around 8 - 12G, but the machine can hold a lot more. Thing is
> memory is still quite expensive, and so I don't to over spec it if
> its not going to get used.
>
> Disk : Ok so this is the main bottleneck of the system. And the
> thing I know least about, so need the most help with. I understand
> you get good improvements if you keep the transaction log on a
> different disk from the database, and that raid 5 is not as good as
> people think unless you have lots of disks.
>
> My option in disks is either 5 x 15K rpm disks or 8 x 10K rpm disks
> (all SAS), or if I pick a different server I can have 6 x 15K rpm
> or 8 x 10K rpm (again SAS). In each case controlled by a PERC 5/i
> (which I think is an LSI Mega Raid SAS 8408E card).
>
You mentioned a "Perc" controller, so I'll assume this is a Dell.

My advice is to find another supplier. check the archives for Dell.

Basically you have no idea what the Perc controller is since it is
whatever Dell decides to ship that day.

In general though you are going down the right path here. Disks
first, memory second, cpu third

Dave

> So the question here is will more disks at a slower speed be better
> than fewer disks as a higher speed?
>
> Assuming I was going to have a mirrored pair for the O/S and
> transaction logs that would leave me with 3 or 4 15K rpm for the
> database, 3 would mean raid 5 (not great at 3 disks), 4 would give
> me raid 10 option if I wanted it.  Or I could have raid 5 across
> all 5/6 disks and not separate the transaction and database onto
> different disks. Better performance from raid 5 with more disks,
> but does having the transaction logs and database on the same disks
> counteract / worsen the performance?
>
> If I had the 8 10K disks, I could have 2 as a mirrored pair for O/S
> Transaction, and still have 6 for raid 5. But the disks are slower.
>
> Anybody have any good thoughts on my disk predicament, and which
> options will serve me better.
>
> Your thoughts are much appreciated.
>
> Regards
>
> Ben
>
>
>
>
>
>
>
> ---------------------------(end of
> broadcast)---------------------------
> TIP 9: In versions below 8.0, the planner will ignore your desire to
>       choose an index scan if your joining column's datatypes do not
>       match
>

Cheers Shane,

> Sounds like you have a very good idea of what to expect. Are these
> solid stats or certain estimates? Estimates can vary when it comes
> time to start.

The figures all come from how my application interacts with the
database when an event happens, so the scaling of operations to each
other is accurate, the number of operations is based on an estimate
of the user interactions with the system, and the figures I quote are
actually peak figures based on some fairly reliable research. If
anything its more likely to be lower then higher, but I like to air
on the side of caution, and so its important for know that I can
sustain this throughput, and have an easy upgrade path in the
hardware I choose now to help if I do need to be able to cope with
more load in the future.

Although I suspect the next step would be to move things like the
logging into a separate database to relieve some of the load.

> I would think 2 will cope with what you describe but what about in
> 12 months time? Can you be sure your needs won't increase? And will
> the cost of 4 CPU's cut your other options? If all 50 users may be
> running the 3rd part at the same time (or is that your 50 trans. a
> second?) then I'd consider the 4.

The 50 connections is pretty much a constant from the distributes
application servers, and only some about 10 of them will be
responsible for running the transactions , the others being more
related to the reading, and logging, and thus mainly staying in the
idle state. So I would think I am better off keeping the CPU sockets
spare, and adding them if needed. Thus enabling more budget for
memory / disks.

> 8GB is a good starting point for a busy server but a few hundred $
> on the extra ram can make more difference than extra disks (more
> for the reading part than writing).

I guess any spare budget I have after the disks should be spend on as
much memory as possible.

> What you describe plans several times 300 inserts to logging plus
> 150 inserts and 50 updates and 1 read a second plus occasional
> reads to the logging and user data.
> Will it be raw data fed in and saved or will the server be
> calculating a majority of the inserted data? If so go for the 4 cpu's.

The inserts are all raw (pre calculated) data, so not work needed by
the database server its self bar the actual insert.

> Generally more disks at slower speed - 2 10K disks in raid 0 is
> faster than 1 15K disk. More disks also allow more options.

Yes I figured striped slow disks are faster then non striped fast
disks, but what about 8 striped slow disks vs 5 striped fast disks?
How do you calculate what the maximum throughput of a disk system
would be? I know that a bit academic really as I need to split the
disks up for the transfer log and the table data, so the large number
of slower disks is as you suggest better anyway.

> I might consider RAID 5 with 8 disks but would lean more for 2 RAID
> 10 setups. This can give you the reliability and speed with system
> and xlog on one and data on the other.

Assuming I go with 8 disks, I guess the real question I have no idea
about is the speed relationship of the transfer log to the table
space data. In other words if I have 2 disks in a raid 1 mirrored
pair for the transfer log (and the O/S, but can't see it needing to
use disk once boots really - so long as it does not need swap space)
and 6 disks in a raid 1 + 0 striped mirrored pair would that be
better than having 2 equal raid 1 + 0 sets of 4 disks.

Clearly if the requirements on the transfer log are the same as the
table data then 2 equal 1+0 sets are better, but if the table data is
at least 1/3 more intensive that the transfer log I think the 2 + 6
should be better. Does anybody know which it is?

> Sounds to me like you have it worked out even if you are a little
> indecisive on a couple of finer points.

Thanks, I guess its more about validating my thoughts are more or
less right, and helping tweak the bits that could be better.

Regards

Ben

> You mentioned a "Perc" controller, so I'll assume this is a Dell.
>
> My advice is to find another supplier. check the archives for Dell.
>
> Basically you have no idea what the Perc controller is since it is
> whatever Dell decides to ship that day.
>
> In general though you are going down the right path here. Disks
> first, memory second, cpu third
>
> Dave

Yes I am looking at either the 2950 or the 6850. I think the only
think that the 6850 really offers me over the 2950 is more
expandability in the spare processor, and additional memory
sockets. In all other respects the config I am looking at would fit
either chassis. Although the 2950, being slightly newer has the DRAC
5 (dells implementation of IPMI) management, which may be useful.

I hear what you say about the raid card, but how likely are they to
change it from the LSI Mega Raid one in reality? But I am open to
suggestions if you have any specific models from other manufacturers
I should look at. I do need to be able to get the fast hardware
support on it though that I can get from the likes of Dells 4 hours
on site call out, so rolling my own isn't an option on this one
really (unless it was so much cheaper I could have a hot standby or
at least a cupboard of all the needed parts instantly available to me)

Regards

Ben

Ben Suffolk wrote:
>> You mentioned a "Perc" controller, so I'll assume this is a Dell.
>>
>> My advice is to find another supplier. check the archives for Dell.
>>
>> Basically you have no idea what the Perc controller is since it is
>> whatever Dell decides to ship that day.
>>
>> In general though you are going down the right path here. Disks first,
>> memory second, cpu third
>>
>> Dave
>
> Yes I am looking at either the 2950 or the 6850. I think the only think
> that the 6850 really offers me over the 2950 is more expandability in
> the spare processor, and additional memory
> sockets. In all other respects the config I am looking at would fit
> either chassis. Although the 2950, being slightly newer has the DRAC 5
> (dells implementation of IPMI) management, which may be useful.

Get an HP with the 64* series. They are a good, well rounded machine for
PostgreSQL.


http://h10010.www1.hp.com/wwpc/pscmisc/vac/us/en/ss/proliant/proliant-dl.html?jumpid=re_R295_prodexp/busproducts/computing-server/proliant-dl

> I hear what you say about the raid card, but how likely are they to
> change it from the LSI Mega Raid one in reality? But I am open to

Heh... very likely. I have a 6 drive Dell machine with a Perc controller
(lsi rebrand). If I put it in RAID 5, it refuses to get more than 8 megs
a second. If I put it in RAID 10, it get about 50 megs a second.

If I get the offshelf LSI Megaraid withe the same configuration? You
don't want to know... it will just make you want to cry at the fact that
you bought a Dell.

Sincerely,

Joshua D. Drake



> suggestions if you have any specific models from other manufacturers I
> should look at. I do need to be able to get the fast hardware support on
> it though that I can get from the likes of Dells 4 hours on site call
> out, so rolling my own isn't an option on this one really (unless it was
> so much cheaper I could have a hot standby or at least a cupboard of all
> the needed parts instantly available to me)
>
> Regards
>
> Ben
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Have you searched our list archives?
>
>               http://archives.postgresql.org
>


--

   === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
   Providing the most comprehensive  PostgreSQL solutions since 1997
             http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

>> Yes I am looking at either the 2950 or the 6850. I think the only
>> think that the 6850 really offers me over the 2950 is more
>> expandability in the spare processor, and additional memory
> I see (in first mail) you plan to use bsd 6.1 on dell2950.
> --- flame on
> Off topic for postgresql performance , but i'd like to warn you
> neither perc5i  crap nor network adapter got proper support for bsd
> 6.1 stable ( dell2950 box )
> dmesg -a | grep bce
> bce0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
>    inet6 fe80::213:72ff:fe61:2ef6%bce1 prefixlen 64 tentative
> scopeid 0x2
> bce0: link state changed to UP
> bce0: /usr/src/sys/dev/bce/if_bce.c(5032): Watchdog timeout
> occurred, resetting!
> bce0: link state changed to DOWN
> bce0: link state changed to UP
> uname -a
> FreeBSD xxx 6.1-STABLE FreeBSD 6.1-STABLE #0:   xxx:/usr/obj/usr/
> src/sys/customkenelcompiled-30-Aug-2006  i386
> Problem with (latest?) raid perc is that only one logical volume is
> supported.
> You may find some bits of info on freebsd mailing lists.
> At least for n/w card problem i see no solution until now.
> 3 month old history:  due to buggy firmware on maxtor disks sold by
> dell 2 servers from our server farm having raid5 crashed and data
> on raid array was lost.
> We were lucky to have proper replication solution.
> If you decide to choose 2950, you have to use linux instead of bsd
> 6.1 . Also buy 2 boxes instead of 1 and set up slony replication
> for redundancy.
> go dell , go to hell.
> --- flame off
>
> good luck!

Thanks Alvis, its good to hear this sort of problem before one
commits to a purchase decision!

I guess it makes the HP's Joshua mentioned in a reply more promising.
Are there any other suppliers I should be looking at do you think.
I'm keen on FreeBSD to be honest rather than Linux (I don't want to
start any holy wars on this as its not the place) as then its the
same as all my other servers, so support / sysadmin is easier if they
are all the same.

How about the Fujitsu Siemens Sun Clones? I have not really looked at
them but have heard the odd good thing about them.

Ben

Hi Ben ,


>> You mentioned a "Perc" controller, so I'll assume this is a Dell.
>>
>> My advice is to find another supplier. check the archives for Dell.
>>
>> Basically you have no idea what the Perc controller is since it is
>> whatever Dell decides to ship that day.
>>
>> In general though you are going down the right path here. Disks
>> first, memory second, cpu third
>>
>> Dave
>
> Yes I am looking at either the 2950 or the 6850. I think the only
> think that the 6850 really offers me over the 2950 is more
> expandability in the spare processor, and additional memory
I see (in first mail) you plan to use bsd 6.1 on dell2950.
--- flame on
Off topic for postgresql performance , but i'd like to warn you neither
perc5i  crap nor network adapter got proper support for bsd 6.1 stable (
dell2950 box )
dmesg -a | grep bce
bce0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
    inet6 fe80::213:72ff:fe61:2ef6%bce1 prefixlen 64 tentative scopeid 0x2
bce0: link state changed to UP
bce0: /usr/src/sys/dev/bce/if_bce.c(5032): Watchdog timeout occurred,
resetting!
bce0: link state changed to DOWN
bce0: link state changed to UP
uname -a
FreeBSD xxx 6.1-STABLE FreeBSD 6.1-STABLE #0:
xxx:/usr/obj/usr/src/sys/customkenelcompiled-30-Aug-2006  i386
Problem with (latest?) raid perc is that only one logical volume is
supported.
You may find some bits of info on freebsd mailing lists.
At least for n/w card problem i see no solution until now.
3 month old history:  due to buggy firmware on maxtor disks sold by dell
2 servers from our server farm having raid5 crashed and data on raid
array was lost.
We were lucky to have proper replication solution.
If you decide to choose 2950, you have to use linux instead of bsd 6.1 .
Also buy 2 boxes instead of 1 and set up slony replication for redundancy.
go dell , go to hell.
--- flame off

good luck!

regards, alvis

> -----Original Message-----
> From: pgsql-performance-owner@postgresql.org
[mailto:pgsql-performance-
> owner@postgresql.org] On Behalf Of Joshua D. Drake
> Sent: Friday, October 20, 2006 2:52 PM
> To: Ben Suffolk
> Cc: Dave Cramer; pgsql-performance@postgresql.org
> Subject: Re: [PERFORM] New hardware thoughts
>
> Ben Suffolk wrote:
> >> You mentioned a "Perc" controller, so I'll assume this is a Dell.
> >>
> >> My advice is to find another supplier. check the archives for Dell.
> >>
> >> Basically you have no idea what the Perc controller is since it is
> >> whatever Dell decides to ship that day.
> >>
> >> In general though you are going down the right path here. Disks
first,
> >> memory second, cpu third
> >>
> >> Dave
> >
> > Yes I am looking at either the 2950 or the 6850. I think the only
think
> > that the 6850 really offers me over the 2950 is more expandability
in
> > the spare processor, and additional memory
> > sockets. In all other respects the config I am looking at would fit
> > either chassis. Although the 2950, being slightly newer has the DRAC
5
> > (dells implementation of IPMI) management, which may be useful.
>
> Get an HP with the 64* series. They are a good, well rounded machine
for
> PostgreSQL.
>
> http://h10010.www1.hp.com/wwpc/pscmisc/vac/us/en/ss/proliant/proliant-
>
dl.html?jumpid=re_R295_prodexp/busproducts/computing-server/proliant-dl
>
> > I hear what you say about the raid card, but how likely are they to
> > change it from the LSI Mega Raid one in reality? But I am open to
>
> Heh... very likely. I have a 6 drive Dell machine with a Perc
controller
> (lsi rebrand). If I put it in RAID 5, it refuses to get more than 8
megs
> a second. If I put it in RAID 10, it get about 50 megs a second.
>
> If I get the offshelf LSI Megaraid withe the same configuration? You
> don't want to know... it will just make you want to cry at the fact
that
> you bought a Dell.

I agree there's better platforms out there than Dell, but the above is
simply not true for the 2950. Raid 5, dd, on 6 disks, I get about
260Mb/s sustained writes. Granted, this should be faster, but... it's a
far cry from 8 or 50MB/s. I posted some numbers here a while back on the
2950, so you might want to dig those out of the archives.

For CPU, if that's a concern, make sure you get Woodcrest with 4MB
shared cache per socket. These are extremely fast CPU's (Intel's 80%
performance improvements over the old Xeons actually seem close). Oh,
and I would NOT recommend planning to add CPU's to a dell box after
you've purchased it. I've seen too many CPU upgrades go awry. Adding
disks, no biggie, adding ram, eh, don't mind, adding CPU, I try to stay
away from for reliability purposes.

Also, I have had experience with at least half dozen 2850's and 2950's -
all have had the LSI controllers re-branded as Perc. If this is a
concern, talk with dell, and I believe you get a 30 day money-back
guarantee. I've used this before, and yes, they will take the server
back. The sales guys aren't too bright, they'll promise anything, but as
long as you can give the server back... (true, we buy a lot of dell
servers.. so... get confirmation from dell on what return policy applies
to your purchase)

If you're not concerned about space, go for the 8 2.5" disks. You'll get
more raw storage out of 300GB 3.5", but unless you need it, you'd be
better served with the additional spindles.

As for FreeBSD- I'd advise taking a good look at 6.2, its' in beta and
they've fixed quite a few problems with the 2950 (Raid controller and
bce nic issues come to mind).

Lastly, if you have the money and rack space for an external disk cage,
take a look at Dell's MD1000 - not as good as some of the sun offerings,
but not too shabby for dell. (Note that I have not tested the MD1000 so
I'm just going off of my 2950 experience and the specs for the MD1000).

The above comes from being stuck with dell and trying to make the best
of it. Turns out it's not as bad as it used to be. Oh, and side note,
this may be obvious for some, but if you're running BSD and need
support, ask to speak to the Linux guys (or simply tell them you're
running Linux). Avoid Dell's windows support at all costs...

- Bucky

On 20-10-2006 16:58 Dave Cramer wrote:
> Ben,
>
>> My option in disks is either 5 x 15K rpm disks or 8 x 10K rpm disks
>> (all SAS), or if I pick a different server I can have 6 x 15K rpm or 8
>> x 10K rpm (again SAS). In each case controlled by a PERC 5/i (which I
>> think is an LSI Mega Raid SAS 8408E card).
>>
> You mentioned a "Perc" controller, so I'll assume this is a Dell.
>
> My advice is to find another supplier. check the archives for Dell.
>
> Basically you have no idea what the Perc controller is since it is
> whatever Dell decides to ship that day.

As far as I know, the later Dell PERC's have all been LSI
Logic-controllers, to my knowledge Dell has been a major contributor to
the LSI-Linux drivers...
At least the 5/i and 5/e have LSI-logic controller chips. Although the
5/e is not an exact copy of the LSI Mega raid 8480E, its board layout
and BBU-memory module are quite different. It does share its
functionality however and has afaik the same controller-chip on it.

Currently we're using a Dell 1950 with PERC 5/e connecting a MD1000
SAS-enclosure, filled with 15 36GB 15k rpm disks. And the Dell-card
easily beats an ICP Vortex-card we also connected to that enclosure.

Ow and we do get much more than, say, 8-50 MB/sec out of it. WinBench99
gets about 644MB/sec in sequential reading tops from a 14-disk raid10
and although IOmeter is a bit less dramatic it still gets over
240MB/sec. I have no idea how fast a simple dd would be and have no
bonnie++ results (at hand) either.
At least in our benchmarks, we're convinced enough that it is a good
set-up. There will be faster set-ups, but at this price-point it won't
surprise me if its the fastest disk-set you can get.

By the way, as far as I know, HP offers the exact same broadcom network
chip in their systems as Dell does... So if that broadcom chip is
unstable on a Dell in FreeBSD, it might very well be unstable in a HP too.

Best regards,

Arjen

>> If I get the offshelf LSI Megaraid withe the same configuration? You
>> don't want to know... it will just make you want to cry at the fact
> that
>> you bought a Dell.
>
> I agree there's better platforms out there than Dell, but the above is
> simply not true for the 2950. Raid 5, dd, on 6 disks, I get about
> 260Mb/s sustained writes. Granted, this should be faster, but... it's a
> far cry from 8 or 50MB/s. I posted some numbers here a while back on the
> 2950, so you might want to dig those out of the archives.

Well these are 3 year old machines, they could have improved a bit but
it is quite true for the version of the Dells I have. I can duplicate it
on both machines.

Frankly Dell has a *long* way to go to prove to me that they are a
quality vendor for Server hardware.

Joshua D. Drake

>

On 20-10-2006 22:33 Ben Suffolk wrote:
> How about the Fujitsu Siemens Sun Clones? I have not really looked at
> them but have heard the odd good thing about them.

Fujitsu doesn't build Sun clones! That really is insulting for them ;-)
They do offer Sparc-hardware, but that's a bit higher up the market.

On the other hand, they also offer nice x86-server hardware. We've had
our hands on a RX300 (2U, dual woodcrest, six 3.5" sas-bays, integraded
lsi-logic raid-controller) and found it to be a very nice machine.

But again, they also offer (the same?) Broadcom networking on board.
Just like Dell and HP. And it is a LSI Logic sas-controller on board, so
if FBSD has trouble with either of those, its hard to find anything
suitable at all in the market.

Best regards,

Arjen

-logic raid-controller) and found it to be a very nice machine.
>
> But again, they also offer (the same?) Broadcom networking on board.
> Just like Dell and HP. And it is a LSI Logic sas-controller on board,
so
> if FBSD has trouble with either of those, its hard to find anything
> suitable at all in the market.
>
You may want to search the bsd -stable and -hardware archives for
confirmation on this, but I believe the RAID/SAS issues have been fixed
in -stable and 6.2-beta1. The bce0 driver appears to have been fixed
more recently, but it's looking like it'll be fixed for the next round
of beta testing.

With any hardware for a critical server, you need to ensure redundancy
(RAID, etc) and for a critical server, you probably want either an
automatic spare hd failover done by the RAID (the 2950 RAID can be
configured to do this) or an entire spare server/replication solution.
While x86 class dells aren't even in the same ballpark as say an IBM
iSeries/pSeries for reliability, I haven't found their more recent boxes
(2850, 2950) to be significantly worse than other vendors (HP might be a
little better, but it's still x86 class hardware).

HTH
- Bucky

On Oct 20, 2006, at 10:58 AM, Dave Cramer wrote:

> My advice is to find another supplier. check the archives for Dell.

Not necessarily bad to go with Dell.  There are *some* of their
controllers that are wicked fast in some configurations.  However,
finding which ones are fast is very tricky unless you buy + return
the box you want to test :-)

>
> Basically you have no idea what the Perc controller is since it is
> whatever Dell decides to ship that day.

FUD!!!

They don't randomly change the controllers under the same name.  If
you order a PERC4e/Si controller you will get the same controller
every time.  This particular controller (found in their PE1850) is
incredibly fast, sustaining over 80Mb/sec writes to a mirror.  I
measured that during a DB mirror using slony.

>
> FUD!!!
>
> They don't randomly change the controllers under the same name.  If you
> order a PERC4e/Si controller you will get the same controller every
> time.

Actually Vivek this isn't true. Yes the hardware will likely be the
same, but the firmware rev will likely be different and I have seen
firmware make an incredible difference for them.

> This particular controller (found in their PE1850) is incredibly
> fast, sustaining over 80Mb/sec writes to a mirror.  I measured that
> during a DB mirror using slony.

O.k. but my experience shows that mirroring isn't where their problem
is, raid 5 or 10 is :)

Sincerely,

Joshua D. Drake




--

   === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
   Providing the most comprehensive  PostgreSQL solutions since 1997
             http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

On Oct 23, 2006, at 5:08 PM, Joshua D. Drake wrote:

>>
>> They don't randomly change the controllers under the same name.
>> If you
>> order a PERC4e/Si controller you will get the same controller every
>> time.
>
> Actually Vivek this isn't true. Yes the hardware will likely be the
> same, but the firmware rev will likely be different and I have seen
> firmware make an incredible difference for them.

Fair enough... but you don't expect LSI to never update their
firmware either, I suspect... not that I'm a big dell apologist..
they're totally off of my personally approved server vendor for db
servers.

>
>> This particular controller (found in their PE1850) is incredibly
>> fast, sustaining over 80Mb/sec writes to a mirror.  I measured that
>> during a DB mirror using slony.
>
> O.k. but my experience shows that mirroring isn't where their problem
> is, raid 5 or 10 is :)

Like I said, for some configurations they're great!  Finding those
configs is difficult.

Vivek Khera wrote:
>
> On Oct 23, 2006, at 5:08 PM, Joshua D. Drake wrote:
>
>>>
>>> They don't randomly change the controllers under the same name.  If you
>>> order a PERC4e/Si controller you will get the same controller every
>>> time.
>>
>> Actually Vivek this isn't true. Yes the hardware will likely be the
>> same, but the firmware rev will likely be different and I have seen
>> firmware make an incredible difference for them.
>
> Fair enough... but you don't expect LSI to never update their firmware
> either, I suspect...

True, but I have *never* had to update the firmware of the LSI (which
was my actual point :))

> Like I said, for some configurations they're great!  Finding those
> configs is difficult.

Agreed.

Sincerely,

Joshua D. Drake


>
>


--

   === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
   Providing the most comprehensive  PostgreSQL solutions since 1997
             http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

On Sat, Oct 21, 2006 at 12:12:59AM +0930, Shane Ambler wrote:
> Generally more disks at slower speed - 2 10K disks in raid 0 is faster
> than 1 15K disk. More disks also allow more options.

Not at writing they're not (unless you're using RAID0... ugh).
--
Jim Nasby                                            jim@nasby.net
EnterpriseDB      http://enterprisedb.com      512.569.9461 (cell)