Re: random_page_costs - are defaults of 4.0 realistic for SCSI RAID 1

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Gregory Stark wrote:
> "Jean-David Beyer" <jeandavid8@verizon.net> writes:
>
>> Gregory Stark wrote (in part):
>>
>>> The extra spindles speed up sequential i/o too so the ratio between sequential
>>> and random with prefetch would still be about 4.0. But the ratio between
>>> sequential and random without prefetch would be even higher.
>>>
>> I never figured out how extra spindles help sequential I-O because
>> consecutive logical blocks are not necessarily written consecutively in a
>> Linux or UNIX file system. They try to group a bunch (8 512-bit?) of blocks
>> together, but that is about it. So even if you are reading sequentially, the
>> head actuator may be seeking around anyway.
>
> That's somewhat true but good filesystems group a whole lot more than 8 blocks
> together. You can do benchmarks with dd and compare the speed of reading from
> a file with the speed of reading from the raw device. On typical consumer
> drives these days you'll get 50-60MB/s raw and I would expect not a whole lot
> less than that with a large ext2 file, at least if it's created all in one
> chunk on a not overly-full filesystem.

# date; dd if=/dev/sda8 of=/dev/null;date
Tue Sep 11 14:27:36 EDT 2007
8385867+0 records in
8385867+0 records out
4293563904 bytes (4.3 GB) copied, 71.7648 seconds, 59.8 MB/s
Tue Sep 11 14:28:48 EDT 2007

# date; dd bs=8192 if=/dev/sda8 of=/dev/null;date
Tue Sep 11 14:29:15 EDT 2007
524116+1 records in
524116+1 records out
4293563904 bytes (4.3 GB) copied, 68.2595 seconds, 62.9 MB/s
Tue Sep 11 14:30:23 EDT 2007

# date; dd bs=8192
if=/srv/dbms/dataA/pgsql/data/pg_xlog/000000010000002B0000002F of=/dev/null;date
Tue Sep 11 14:34:25 EDT 2007
2048+0 records in
2048+0 records out
16777216 bytes (17 MB) copied, 0.272343 seconds, 61.6 MB/s
Tue Sep 11 14:34:26 EDT 2007

The first two are the partition where the W.A.L. is in (and a bit more:

[/srv/dbms/dataA/pgsql/data]# ls -l
total 104
- -rw------- 1 postgres postgres     4 Aug 11 13:32 PG_VERSION
drwx------ 5 postgres postgres  4096 Aug 11 13:32 base
drwx------ 2 postgres postgres  4096 Sep 11 14:35 global
drwx------ 2 postgres postgres  4096 Sep 10 18:58 pg_clog
- -rw------- 1 postgres postgres  3396 Aug 11 13:32 pg_hba.conf
- -rw------- 1 root     root      3396 Aug 16 14:32 pg_hba.conf.dist
- -rw------- 1 postgres postgres  1460 Aug 11 13:32 pg_ident.conf
drwx------ 4 postgres postgres  4096 Aug 11 13:32 pg_multixact
drwx------ 2 postgres postgres  4096 Sep 10 19:48 pg_subtrans
drwx------ 2 postgres postgres  4096 Aug 12 16:14 pg_tblspc
drwx------ 2 postgres postgres  4096 Aug 11 13:32 pg_twophase
drwx------ 3 postgres postgres  4096 Sep 10 19:53 pg_xlog
- -rw------- 1 postgres postgres 15527 Sep  8 00:35 postgresql.conf
- -rw------- 1 postgres postgres 13659 Aug 11 13:32 postgresql.conf.dist
- -rw------- 1 root     root     15527 Sep  4 10:37 postgresql.conf~
- -rw------- 1 postgres postgres    56 Sep  8 08:12 postmaster.opts
- -rw------- 1 postgres postgres    53 Sep  8 08:12 postmaster.pid

It is tricky for me to find a big enough file to test. I tried one of the
pg_xlog files, but I cannot easily copy from there because it acts a bit
interactive and the time is mostly my time. If I copy it elsewhere and give
it to non-root, then it is all in the cache, so it does not really read it.
>
> (Those assumptions is not necessarily valid for Postgres which is another
> topic, but one that requires some empirical numbers before diving into.)
>


- --
  .~.  Jean-David Beyer          Registered Linux User 85642.
  /V\  PGP-Key: 9A2FC99A         Registered Machine   241939.
 /( )\ Shrewsbury, New Jersey    http://counter.li.org
 ^^-^^ 14:30:04 up 33 days, 17:52, 1 user, load average: 5.50, 4.67, 4.29
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)
Comment: Using GnuPG with CentOS - http://enigmail.mozdev.org

iD8DBQFG5uM4Ptu2XpovyZoRAhtlAKDFs5eP/CGIqB/z207j2dpwDSHOlwCfevp4
lBWn3b2GW6gesaq+l3Rbooc=
=F4H6
-----END PGP SIGNATURE-----