Re: concurrent IO in postgres?

От: John W Strange
Тема: Re: concurrent IO in postgres?
Дата: ,
(см: обсуждение, исходный текст)
Ответ на: concurrent IO in postgres?  (Przemek Wozniak)
Ответы: Re: concurrent IO in postgres?  ("Kevin Grittner")
Re: concurrent IO in postgres?  (Andy)
Список: pgsql-performance

Скрыть дерево обсуждения

concurrent IO in postgres?  (Przemek Wozniak, )
 Re: concurrent IO in postgres?  (Scott Marlowe, )
  Re: concurrent IO in postgres?  (Przemek Wozniak, )
   Re: concurrent IO in postgres?  (Jeff Janes, )
    Re: concurrent IO in postgres?  (Mladen Gogala, )
     Re: concurrent IO in postgres?  (Jeff Janes, )
      Re: concurrent IO in postgres?  (Greg Smith, )
 Re: concurrent IO in postgres?  (John W Strange, )
  Re: concurrent IO in postgres?  ("Kevin Grittner", )
  Re: concurrent IO in postgres?  (Andy, )
   Re: concurrent IO in postgres?  (Ben Chobot, )
 Re: concurrent IO in postgres?  (John Cagle, )
  Re: concurrent IO in postgres?  (Josh Berkus, )
   Re: concurrent IO in postgres?  ("Pierre C", )

Typically my problem is that the large queries are simply CPU bound..  do you have a sar/top output that you see. I'm
currentlysetting up two FusionIO DUO @640GB in a lvm stripe to do some testing with, I will publish the results after

If anyone has some tests/suggestions they would like to see done please let me know.

- John

-----Original Message-----
From:  [mailto:] On Behalf Of Przemek
Sent: Thursday, December 23, 2010 11:38 AM
Subject: [PERFORM] concurrent IO in postgres?

When testing the IO performance of ioSAN storage device from FusionIO
(650GB MLC version) one of the things I tried is a set of IO intensive
operations in Postgres: bulk data loads, updates, and queries calling
for random IO. So far I cannot make Postgres take advantage of this
tremendous IO capacity. I can squeeze a factor of a few here and there
when caching cannot be utilized, but this hardware can do a lot more.

Low level testing with fio shows on average x10 speedups over disk for
sequential IO and x500-800 for random IO. With enough threads I can get
IOPS in the 100-200K range and 1-1.5GB/s bandwidth, basically what's
advertised. But not with Postgres.

Is this because the Postgres backend is essentially single threaded and
in general does not perform asynchronous IO, or I'm missing something?
I found out that the effective_io_concurrency parameter only takes
effect for bitmap index scans.

Also, is there any work going on to allow concurrent IO in the backend
and adapt Postgres to the capabilities of Flash?

Will appreciate any comments, experiences, etc.

Przemek Wozniak

Sent via pgsql-performance mailing list ()
To make changes to your subscription:
This communication is for informational purposes only. It is not
intended as an offer or solicitation for the purchase or sale of
any financial instrument or as an official confirmation of any
transaction. All market prices, data and other information are not
warranted as to completeness or accuracy and are subject to change
without notice. Any comments or statements made herein do not
necessarily reflect those of JPMorgan Chase & Co., its subsidiaries
and affiliates.

This transmission may contain information that is privileged,
confidential, legally privileged, and/or exempt from disclosure
under applicable law. If you are not the intended recipient, you
are hereby notified that any disclosure, copying, distribution, or
use of the information contained herein (including any reliance
thereon) is STRICTLY PROHIBITED. Although this transmission and any
attachments are believed to be free of any virus or other defect
that might affect any computer system into which it is received and
opened, it is the responsibility of the recipient to ensure that it
is virus free and no responsibility is accepted by JPMorgan Chase &
Co., its subsidiaries and affiliates, as applicable, for any loss
or damage arising in any way from its use. If you received this
transmission in error, please immediately contact the sender and
destroy the material in its entirety, whether in electronic or hard
copy format. Thank you.

Please refer to for
disclosures relating to European legal entities.

В списке pgsql-performance по дате сообщения:

От: Ben Chobot
Сообщение: Re: concurrent IO in postgres?
От: John Cagle
Сообщение: Re: concurrent IO in postgres?