Обсуждение: Separate volumes

I was once told that it's best practice to store tables and indexes in separate tablespaces located on separate physical drives. It seemed logical that this should improve performance because the read-head wouldn't need to jump back and forth between a table and its index. 

However, I can't seem to find this advice anywhere online. Is it indeed best practice? Is it worth the hassle?

          -Ed

Ed Behn | Senior Systems Engineer | Avionics

COLLINS ÆROSPACE

2551 Riva Road, Annapolis, MD 21401 USA

Tel: +1 410 266 4426 | Mobile: +1 240 696 7443

ed.behn@collins.com | collinsaerospace.com

CONFIDENTIALITY WARNING: This message may contain proprietary and/or privileged information of Collins Aerospace and its affiliated companies. If you are not the intended recipient, please 1) Do not disclose, copy, distribute or use this message or its contents. 2) Advise the sender by return email. 3) Delete all copies (including all attachments) from your computer. Your cooperation is greatly appreciated.

Hi Steve,

Coming from oracle land, tablespaces play a bigger role than they do in PG land.  In PG land, they can control the mapping of tables/indexes to faster or slower devices. By separating a table's tablespace from its index tablespace, you may get more parallel I/O.  They also allow for flexibility in setting pg config parameters per tablespace:

alter tablespace mytablespace set ( seq_page_cost=0.5, random_page_cost=0.5 );

But they also can be a headache in managing stuff.   For instance, all replicas must have the same directory structure and symlinks.

Regards,
Michael Vitale



Steve Midgley wrote on 4/6/2020 1:11 PM:

On Mon, Apr 6, 2020 at 9:42 AM Erik Brandsberg <erik@heimdalldata.com> wrote:

With SSD and it's random IO performance, I doubt that this advice would apply as much, and adds complexity to your configuration and management.  In particular if you use any filesystem level snapshotting (like with ZFS), splitting the filespaces will make it harder to do restores and using snapshots.

On Mon, Apr 6, 2020 at 10:40 AM Ed Behn <ed.behn@collins.com> wrote:

I was once told that it's best practice to store tables and indexes in separate tablespaces located on separate physical drives. It seemed logical that this should improve performance because the read-head wouldn't need to jump back and forth between a table and its index. 

However, I can't seem to find this advice anywhere online. Is it indeed best practice? Is it worth the hassle?

 

As a general and practical matter I 100% agree with Erik -- the advice is a bit out of date, and for SSDs it probably makes no meaningful difference. However for extremely high, sustained workloads, you might find splitting tables, indices, and transaction logs onto separate disk _disk arrays and controllers_ could yield improvements, particularly for certain RAID setups. But maxing out a disk controller is pretty hard to do (impossible afaik with a single drive), so you'd want to have some strong metrics to show this is worth it. At that point, you'd probably be better off getting commercial disk array solutions into the mix rather than rolling your own anyway..

Steve

On Tue, Apr  7, 2020 at 04:31:51PM +0000, Lu Dillon wrote:
> Hi All,
>
> This is a very intersting question. I believe this is not just a best practice
> to PG. We can apply to all RDBMS. In my opinion, I agree with the others: with
> SSD, you don’t separate tables and indexs to different disks. I think the IOPS
> is enough. If you still have a problem of IOPS, you can try NVME device or U2
> device.

If you are mixing magnetic and SSDs for the same database, having
indexes on SSDs can really help, compared to table files on SSDs, where
the benefit is more limited.  Also, having current data on SSDs and
archive data on magnetic is also useful, and you usually use
time-based partitioning for such cases.

--
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+                      Ancient Roman grave inscription +

On Fri, Apr 10, 2020 at 04:52:06PM -0400, Erik Brandsberg wrote:
> A modern filesystem can help avoid even this complexity.  As an example, I am
> managing one PG setup that is self-hosted on an AWS EC2 instance, with 16TB of
> raw storage.  The bulk of that storage is in ST1, or the cheapest rotating disk
> capacity available in EBS, but is using ZFS as the filesystem (with
> compression, so realistically about 35TB of raw data).  The instance type is a
> Z1d.metal, which has two 900GB NVME drives, which have been divided to provide
> swap space, as well as ZFS read and write caching.  This setup has largely
> offset the slow performance of the st1 disks, and kept the performance usable
> (most of the data is legacy, and rarely used).   I'm a big fan of keeping the
> DB configuration simple, as it is way too easy to overlook tuning of a
> filespace for an index, causing performance problems, while if you keep it
> auto-tuning at the filesystem level, it "just works".

You are saying the cloud automatically moves data between the fast and
slow storage?  I know many NAS systems do this, but I have also seen
problems when NAS systems guess wrong.

--
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+                      Ancient Roman grave inscription +