Обсуждение: Specifications for a new server

On Tue, May 06, 2014 at 11:13:42AM +0200, Johann Spies wrote:
>Analysis or the SAR-logs showed that there were too much iowait in the CPU's on
>the old system which has a lower spec CPU than the ones considered for the new
>system.

iowait means the cpu is doing nothing but waiting for data from the
disk. buying faster cpus means that they will be able to spend more time
waiting for data from the disk. you'd probably get much better bang for
the buck upgrading the storage subsystem than throwing more money at
cpus.

>We are looking possibly the following hardware:
>
>CPU: 2 x  Ivy Bridge 8C E5-2667V2 3.3G 25M 8GT/s QPI - 16 cores
>RAM: 24 x 32GB DDR3-1866 2Rx4 LP ECC REG RoHS  - 768Gb
>
>with enough disk space - about 4.8 Tb on RAID 10.
>My question is about the possible advantage and usage of SSD disks in the new
>server. 

>At the moment I am considering using 2 x 200GB SSD' s for a separate
>partion for temporary files and 2 x 100GB for the operating system.

If you're talking about SSDs for the OS, that's a complete waste; there
is essentially no I/O relating to the OS once you've booted.

>So my questions:
>
>1. Will the SSD's in this case be worth the cost?
>2.  What will the best way to utilize them in the system?

The best way to utilize them would probably be to spend less on the CPU
and RAM and more on the storage, and use SSD either for all of the
storage or for specific items that have a high level of I/O (such as the
indexes). Can't be more specific than that without a lot more
information about the database, how it is utilized, and what's actually
slow.

Mike Stone

On Tue, May 06, 2014 at 01:15:10PM +0200, Dorian Hoxha wrote:
>Since the commitlog/WAL is sequential-write, does it mattert that much to put
>it in ssd

No, assuming a good storage system with nvram write buffer.

Mike Stone

On 6.5.2014 16:43, Dorian Hoxha wrote:
> And nvram is ram on hardware-raid controller that is not erased on
> reboot(battery) ?

Yes.

> Can this nvram be used also when the configuration is
> jbod(justabunchofdisks) or some kind of raid(0/1/5/6/10 etc) is
> required to use the nvram.

That probably depends on the controller used. Some controllers (e.g.
Dell Perc H700 and such) don't even support JBOD - an easy way around
this is create single-drive RAID0 arrays (never tried it, though).

Why do you even want to use JBOD?

regards
Tomas

Hello,

here I am - the lonely user of JBOD :-) I use exactly what you describe below, we have a couple of Dell Servers with
batterybacked H700 controllers. On one we let the controller do a RAID10 on others I use the single disks (as you
describein useless RAID0s).  

What we do then is use those disks in a Solaris ZFS Pool as filesystem for our databases. The reason is, that with ZFS
(maybethere are other fs around there which can do this, but far back in good old Sun-times it was the only one) I can
buildup a pool and mix nearline SAS with SSDs in a neat way. ZFS has the concept of an ZFS Intent Log (ZIL), where
writesgo first before migrated to the pool - and you can put them on seperate disks. And you have the adaptive
replacementcache (ARC) you can also put on seperate disks, where hot data is cached. Now guess where we use the
expensiveSSDs. 

It boils down to this:

- slow mechanical drives backed with nvram controller
- SSD expecially for the ARC (and ZIL)
- RAM for ZFS further caching

On my very first try when ZIL and ARC approached in Solaris my first idea was just put one SSD splitted to two
partitionsas ARC and ZIL to our exisiting database. I didn't change anything else  (to be fair, that was a development
systemwith out nvram controller) - gave me a boost in performace of about 10. 

To be a bit more fair (I can, as I do not work for Sun - ehm Oracle): If this would work for someone else, the usage
patternshould be kept in mind. Also setting and tuning a Solaris server is of some more work and ZFS itself uses a good
bunchof CPU and RAM (we leave 8G for ZFS) and one would need much testing and tuning. 

A positive effect we use is for backing up data - since ZFS offers atomic snapshots, we just snapshot the filesystem.
Thedatafiles are guaranteed to be vaild as the WAL. A restore is to put the files back into place and restart Postgres
-it will run a recovery and replay the WAL and everything is done. We used that to provide back in time recovery and
warmstandby with old postgres installations before replication was included into core. 

Cheers,
Roland



> Can this nvram be used also when the configuration is
> jbod(justabunchofdisks) or some kind of raid(0/1/5/6/10 etc) is
> required to use the nvram.

That probably depends on the controller used. Some controllers (e.g.
Dell Perc H700 and such) don't even support JBOD - an easy way around
this is create single-drive RAID0 arrays (never tried it, though).

Why do you even want to use JBOD?



----------------------------------------------------------------
E-Mail-Postfach voll? Jetzt kostenlos E-Mail-Adresse @t-online.de sichern und endlich Platz für tausende E-Mails haben.
http://www.t-online.de/email-kostenlos

On Thu, May 08, 2014 at 10:11:38AM +0200, Johann Spies wrote:
>I understand your remark about the CPU in the light of my wrong assumption
>earlier, but I do not understand your remark about the RAM.  The fact that
>temporary files of up to 250Gb are created at times during complex queries, is
>to me an indication of too low RAM.

If you can afford infinite RAM, then infinite RAM is great. If your
working set size exceeds the memory size, then you will eventually need
to deal with disk IO. At that point, maybe a bit more memory will help
and maybe it will not--you'll be able to fit a little bit more working
data into memory, but that won't likely radically change the
performance. (If you can afford to fit *everything* you need into RAM
than that's ideal, but that's not the case for most people with
non-trival data sets.) What is certain is that improving the disk IO
performance will improve your overall performance if you're IO bound.

(And the mere existence of temporary files isn't an indication of
insufficient RAM if the system can utilize the memory more efficiently
with the files than it can without them--they could contain data that
isn't needed in a particular phase of a query, freeing up resources that
are needed for other data in that phase.)

Mike Stone