Обсуждение: how to partition disks

hi
i'm waiting for new server to arrive.
for the server i will get hp msa1000, with 14 discs (72g, 15000 rpm).
what kind of partitioning you suggest, to get maximum performance?
for system things i will have separate discs, so whole array is only for postgresql.

data processing is oltp, but with large amounts of write operations.

any hints?

should i go with 3 partitions, or just 1? or 2?

depesz

--
http://www.depesz.com/ - nowy, lepszy depesz

Hi Hubert,

hubert depesz lubaczewski schrieb:
> hi
> i'm waiting for new server to arrive.
> for the server i will get hp msa1000, with 14 discs (72g, 15000 rpm).
> what kind of partitioning you suggest, to get maximum performance?
> for system things i will have separate discs, so whole array is only for
> postgresql.
>
> data processing is oltp, but with large amounts of write operations.
>
> any hints?
>
> should i go with 3 partitions, or just 1? or 2?
>

You should configure your discs to RAID 10 volumes.
You should set up a separate volume for WAL.
A volume for an additional table space may also useful.

In your case I would do 2 partitions:

1. RAID 10 with 8 discs for general data
2. RAID 10 with 4 discs for WAL
(two disk as spare)

You may split the first volume in two volumes for a second table space
if you Server doesn't have enough RAM and you have a high disk read I/O.


Cheers
Sven.

On 6/14/06, Sven Geisler <sgeisler@aeccom.com> wrote:
raid 10 is of course not questionable. but are you sure that it will work faster than for example:
2 discs (raid 1) for xlog
6 discs (raid 10) for tables
6 discs (raid 10) for indices?

depesz

--
http://www.depesz.com/ - nowy, lepszy depesz

Hi Hupert,

hubert depesz lubaczewski schrieb:
> On 6/14/06, *Sven Geisler* <sgeisler@aeccom.com
> <mailto:sgeisler@aeccom.com>> wrote:
>     You should configure your discs to RAID 10 volumes.
>     You should set up a separate volume for WAL.
>     A volume for an additional table space may also useful.
>     In your case I would do 2 partitions:
>     1. RAID 10 with 8 discs for general data
>
>
> raid 10 is of course not questionable. but are you sure that it will
> work faster than for example:
> 2 discs (raid 1) for xlog
> 6 discs (raid 10) for tables
> 6 discs (raid 10) for indices?
>

This depends on your application. Do you have a lot of disc reads?
Anyhow, I would put the xlog always to a RAID 10 volume because most of
the I/O for update and inserts is going to the xlog.

4 discs xlog
6 discs tables
4 discs tables2

This should be better. You should distribute indices on separate spindle
stacks to share the I/O. But again this depends on your application and
your server. How are the indices used? How large is your file system
cache. What does PostgreSQL effectively read from disc.

Don't forget to tune your postgresql.conf:
<http://www.powerpostgresql.com/PerfList>
<http://www.powerpostgresql.com/Downloads/terabytes_osc2005.pdf>

Cheers
Sven.

> > raid 10 is of course not questionable. but are you sure that it will
> > work faster than for example:
> > 2 discs (raid 1) for xlog
> > 6 discs (raid 10) for tables
> > 6 discs (raid 10) for indices?
> >
>
> This depends on your application. Do you have a lot of disc reads?
> Anyhow, I would put the xlog always to a RAID 10 volume because most of
> the I/O for update and inserts is going to the xlog.
>
> 4 discs xlog
> 6 discs tables
> 4 discs tables2

I have a question in regards to I/O bandwidths of various raid configuration.  Primary, does the
above suggested raid partitions imply that multiple (smaller) disk arrays have a potential for
more I/O bandwidth than a larger raid 10 array?

Regards,

Richard

Hi Richard,

Richard Broersma Jr schrieb:
>> This depends on your application. Do you have a lot of disc reads?
>> Anyhow, I would put the xlog always to a RAID 10 volume because most of
>> the I/O for update and inserts is going to the xlog.
>>
>> 4 discs xlog
>> 6 discs tables
>> 4 discs tables2
>
> I have a question in regards to I/O bandwidths of various raid configuration.  Primary, does the
> above suggested raid partitions imply that multiple (smaller) disk arrays have a potential for
> more I/O bandwidth than a larger raid 10 array?

Yes.
Because the disc arms didn't need to reposition that much as there would
o with one large volume.

For example, You run two queries with two clients and each queries needs
to read some indices from disk. In this case it more efficient to read
from different volumes than to read from one large volume where the disc
arms has to jump.

Sven.

On Wed, Jun 14, 2006 at 04:32:23PM +0200, Sven Geisler wrote:
> Hi Richard,
>
> Richard Broersma Jr schrieb:
> >>This depends on your application. Do you have a lot of disc reads?
> >>Anyhow, I would put the xlog always to a RAID 10 volume because most of
> >>the I/O for update and inserts is going to the xlog.
> >>
> >>4 discs xlog
> >>6 discs tables
> >>4 discs tables2
> >
> >I have a question in regards to I/O bandwidths of various raid
> >configuration.  Primary, does the
> >above suggested raid partitions imply that multiple (smaller) disk arrays
> >have a potential for
> >more I/O bandwidth than a larger raid 10 array?
>
> Yes.
> Because the disc arms didn't need to reposition that much as there would
> o with one large volume.
>
> For example, You run two queries with two clients and each queries needs
> to read some indices from disk. In this case it more efficient to read
> from different volumes than to read from one large volume where the disc
> arms has to jump.

But keep in mind that all of that is only true if you have very good
knowledge of how your data will be accessed. If you don't know that,
you'll almost certainly be better off just piling everything into one
RAID array and letting the controller deal with it.

Also, if you have a good RAID controller that's batter-backed,
seperating pg_xlog onto it's own array is much less likely to be a win.
The reason you normally put pg_xlog on it's own partition is because the
database has to fsync pg_xlog *at every single commit*. This means you
absolutely want that fsync to be as fast as possible. But with a good,
battery-backed controller, this no longer matters. The fsync is only
going to push the data into the controller, and the controller will take
things from there. That means it's far less important to put pg_xlog on
it's own array. I actually asked about this recently and one person did
reply that they'd done testing and found it was better to just put all
their drives into one array so they weren't wasting bandwidth on the
pg_xlog drives.

Even if you do decide to keep pg_xlog seperate, a 4 drive RAID10 for
that is overkill. It will be next to impossible for you to generate
enough WAL traffic to warrent it.

Your best bet is to perform testing with your application. That's the
only way you'll truely find out what's going to work best. Short of
that, your best bet is to just pile all the drives together. If you do
testing, I'd start first with the effect of a seperate pg_xlog. Only
after you have those results would I consider trying to do things like
split indexes from tables, etc.

BTW, you should consider reserving some of the drives in the array as
hot spares.
--
Jim C. Nasby, Sr. Engineering Consultant      jnasby@pervasive.com
Pervasive Software      http://pervasive.com    work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf       cell: 512-569-9461

On Wed, Jun 14, 2006 at 04:32:23PM +0200, Sven Geisler wrote:
>For example, You run two queries with two clients and each queries needs
>to read some indices from disk. In this case it more efficient to read
>from different volumes than to read from one large volume where the disc
>arms has to jump.

Hmm. Bad example, IMO. In the case of reading indices you're doing
random IO and the heads will be jumping all over the place anyway. The
limiting factor there will be seeks/s, and you'll possibly get better
results with the larger array. (That case is fairly complicated to
analyze and depends very much on the data.) Where multiple arrays will be
faster is if you have a lot of sequential IO--in fact, a couple of cheap
disks can blow away a fairly expensive array for purely sequential
operations since each disk can handle >60MB/s of if it doesn't have to
seek, whereas multiple sequential streams on the big array will cause
each disk in the array to seek. (The array controller will try to hide
this with its cache; its cache size & software will determine how
successful it is at doing so.)

Mike Stone

hubert depesz lubaczewski writes:

> On 6/14/06, Sven Geisler
> raid 10 is of course not questionable. but are you sure that it will work
> faster than for example:
> 2 discs (raid 1) for xlog
> 6 discs (raid 10) for tables
> 6 discs (raid 10) for indices?


Caching up on the performance list.
Although this may not help the original poster.. wanted to share a recent
experience related to allocation of disks on a raid.

We just got a server with 16 disks.
We condfigured 12 to 1 raid controller and a second raid with 4. Both using
raid 10.

RAID 1
10 x 7,200rpm disks
2 hot spares

RAID 2
4 x 10,000 rpm disk

One of the things I always do with new machines is to run bonnie++ and get
some numbers.

I expected the second raid to have better numbers than the first because the
disks were 10K drives (all SATA). To my surprise the larger raid had better
numbers.

So I figure the number of spindles on a single RAID does make a big
difference. To that regard splitting 16 disks into 3 sets may help with data
needing to be read/written to be in separate raids, but may degrade
performance by reducing the number of spindles on each of the raids.