Re: Shared buffers, db transactions commited, and write IO on Solaris

Well, to check if there is a real potential gain all we need is a

small comparing test using PgSQL compiled with LOG block size equal to

say 1K and direct IO enabled.

Rgds,

-Dimitri

On 3/30/07, Kenneth Marshall <ktm@rice.edu> wrote:

> On Fri, Mar 30, 2007 at 04:25:16PM +0200, Dimitri wrote:

> > The problem is while your goal is to commit as fast as possible - it's

> > pity to vast I/O operation speed just keeping common block size...

> > Let's say if your transaction modification entering into 512K - you'll

> > be able to write much more 512K blocks per second rather 8K per second

> > (for the same amount of data)... Even we rewrite probably several

> > times the same block with incoming transactions - it still costs on

> > traffic, and we will process slower even H/W can do better. Don't

> > think it's good, no? ;)

> >

> > Rgds,

> > -Dimitri

> >

> With block sizes you are always trading off overhead versus space

> efficiency. Most OS write only in 4k/8k to the underlying hardware

> regardless of the size of the write you issue. Issuing 16 512byte

> writes has much more overhead than 1 8k write. On the light transaction

> end, there is no real benefit to a small write and it will slow

> performance for high throughput environments. It would be better to,

> and I think that someone is looking into, batching I/O.

>

> Ken

>

Folks,

to close topic with "LOG block size=1K" idea - I took a time to test

it (yes) and in best cases there is only 15% gain comparing to 8K -

storage protocol is quite heavy itself, so less or more data sent

within it doesn't reduce service time too much... As well even this

gain is quickly decreasing with growing workload! So, yes 8K is good

enough and probably the most optimal choice for LOG (as well data)

block size.

Rgds,

-Dimitri