Re: Hardware/OS recommendations for large databases (

Поиск
Список
Период
Сортировка
От Bruce Momjian
Тема Re: Hardware/OS recommendations for large databases (
Дата
Msg-id 200511230453.jAN4rGs24421@candle.pha.pa.us
обсуждение исходный текст
Ответ на Re: Hardware/OS recommendations for large databases (  (Alan Stange <stange@rentec.com>)
Список pgsql-performance
Alan Stange wrote:
> Bruce Momjian wrote:
> > Right now the file system will do read-ahead for a heap scan (but not an
> > index scan), but even then, there is time required to get that kernel
> > block into the PostgreSQL shared buffers, backing up Luke's observation
> > of heavy memcpy() usage.
> >
> > So what are our options?  mmap()?  I have no idea.  Seems larger page
> > size does help.

> For sequential scans, you do have a background reader.  It's the
> kernel.  As long as you don't issue a seek() between read() calls, the

I guess you missed my text of "Right now the file system will do
read-ahead", meaning the kernel.

> I don't think the memcpy of data from the kernel to userspace is that
> big of an issue right now.  dd and all the high end network interfaces
> manage OK doing it, so I'd expect postgresql to do all right with it now
> yet too.   Direct IO will avoid that memcpy, but then you also don't get
> any caching of the files in memory.  I'd be more concerned about any
> memcpy calls or general data management within postgresql.    Does
> postgresql use the platform specific memcpy() in libc?  Some care might
> be needed to ensure that the memory blocks within postgresql are all
> properly aligned to make sure that one isn't ping-ponging cache lines
> around (usually done by padding the buffer sizes by an extra 32 bytes or
> L1 line size).   Whatever you do, all the usual high performance
> computing tricks should be used prior to considering any rewriting of
> major code sections.

We have dealt with alignment and MemCpy is what we used for small-sized
copies to reduce function call overhead.  If you want to improve it,
feel free to take a look.

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

В списке pgsql-performance по дате отправления:

Предыдущее
От: Greg Stark
Дата:
Сообщение: Re: Hardware/OS recommendations for large databases (
Следующее
От: Michael Fuhr
Дата:
Сообщение: Re: Stored Procedure