Benchmark with FreeBSD 8.0 and pgbench

This is not a rigorous test and should not be used as a direct comparison
between
operating systems. My objective was to estimate the ZFS toll in Postgres,
and
what to expect in performance, comparing to the old server this machine will
be replacing. Not all possible configurations were tested, and the CentOS
benchmark was made out of curiosity.

All tests were made using the default instalation, without any additional
tuning,
except those mentioned in custom configuration. The installation media used
was the
8.0-STABLE-201002 DVD (amd64). The server was rebooted between configuration
changes, but NOT between successive runs.
There are some inconsistencies between results,probably due to caching.
Disabling atime surprisingly results in slower runs, in both ZFS and UFS.
The CentOS test was made with the same default configuration file used in
FreeBSD.

Any additional hints are welcome, but I probably won't be able to re-run
the tests :)

Regards,
    João Pinheiro


Machine Specs:
 - Hp Proliant ML330
 - 1x Intel(R) Xeon CPU E5504 2.0Ghz
 - 12GB RAM
 - 4 x 250GB SATA HD 7.2 (RAID 10)
 - P410 Raid controller w/512MB cache/battery option


OS and Postgres Custom Configuration (used on tests marked as custom):

 postgresql.conf:
 ----------------------
 max_connections = 500
 work_mem = 16MB
 shared_buffers = 3072MB

 /etc/sysctl.conf
 ---------------------
 kern.ipc.shmmax=3758096384
 kern.ipc.shmall=917504
 kern.ipc.shm_use_phys=1

 /boot/loader.conf
 -----------------------
 kern.ipc.semmni=512
 kern.ipc.semmns=1024
 kern.ipc.semmnu=512

FreeBSD Setup:
 $ uname -a
 FreeBSD beastie 8.0-STABLE-201002 FreeBSD 8.0-STABLE-201002 #0: Tue Feb 16
21:05:59 UTC 2010
     root@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  amd64

CentOS setup:
 [root@dhcppc0 test]# uname -a
 Linux dhcppc0 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009 x86_64
x86_64 x86_64 GNU/Linux
 iptables: off
 selinux: off

-------------------------------------------------------------------------------
FreeBSD 8.0-STABLE, UFS, postgresql84-server from package, default config
-------------------------------------------------------------------------------

pgbench -c 10 -t 1000 testdb : 2363.017640 (including connections
establishing)
pgbench -c 20 -t 1000 testdb : 2310.928229 (including connections
establishing)
pgbench -c 30 -t 1000 testdb : 2042.681002 (including connections
establishing)
pgbench -c 40 -t 1000 testdb : 1891.323153 (including connections
establishing)

bonnie++:

Version  1.96       ------Sequential Output------ --Sequential Input-
--Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec
%CP
beastie         24G   700  99 170476  22 59826  11   967  78 156937  15
475.7   5
Latency             12177us     500ms     830ms     291ms     145ms
269ms
Version  1.96       ------Sequential Create------ --------Random
Create--------
beastie             -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec
%CP
                 20 35329  48 +++++ +++ +++++ +++ 19828  86 +++++ +++ +++++
+++
Latency             26925us      15us      83us     547ms      80us
60us

-------------------------------------------------------------------------------
FreeBSD 8.0-STABLE, UFS,noatime, postgresql84-server from package, default
conf
-------------------------------------------------------------------------------

pgbench -c 10 -t 1000 testdb : 2287.987630 (including connections
establishing)
pgbench -c 20 -t 1000 testdb : 2255.514875 (including connections
establishing)
pgbench -c 30 -t 1000 testdb : 2098.280816 (including connections
establishing)
pgbench -c 40 -t 1000 testdb : 1871.193058 (including connections
establishing)

bonnie++:

Version  1.96       ------Sequential Output------ --Sequential Input-
--Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec
%CP
beastie         24G   679  99 170965  22 58682  11   942  76 157156  15
498.7   6
Latency             12881us     569ms    1171ms     681ms     155ms
242ms
Version  1.96       ------Sequential Create------ --------Random
Create--------
beastie             -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec
%CP
                 20 36118  49 +++++ +++ 27231  99 +++++ +++ +++++ +++ +++++
+++
Latency             32705us      81us     537ms   14280us      25us
82us

-------------------------------------------------------------------------------
FreeBSD 8.0-STABLE, UFS, postgresql84-server from package, custom config
-------------------------------------------------------------------------------

pgbench -c 10 -t 1000 testdb : 2320.509732 (including connections
establishing)
pgbench -c 20 -t 1000 testdb : 2289.853556 (including connections
establishing)
pgbench -c 30 -t 1000 testdb : 2089.112777 (including connections
establishing)
pgbench -c 40 -t 1000 testdb : 1921.034254 (including connections
establishing)
pgbench -c 50 -t 1000 testdb : 1574.231270 (including connections
establishing)
pgbench -c 100 -t 1000 testdb : 1096.761450 (including connections
establishing)
pgbench -c 200 -t 1000 testdb : 256.443268 (including connections
establishing)
pgbench -c 300 -t 1000 testdb : 69.174219 (including connections
establishing)

-------------------------------------------------------------------------------
FreeBSD 8.0-STABLE, UFS, postgresql84-server from ports, default config
-------------------------------------------------------------------------------

pgbench -c 10 -t 1000 testdb : 1676.760766 (including connections
establishing)
pgbench -c 20 -t 1000 testdb : 2295.344502 (including connections
establishing)
pgbench -c 30 -t 1000 testdb : 2066.721058 (including connections
establishing)
pgbench -c 40 -t 1000 testdb : 1887.196064 (including connections
establishing)

-------------------------------------------------------------------------------
FreeBSD 8.0-STABLE, UFS, postgresql84-server from ports, custom config
-------------------------------------------------------------------------------

pgbench -c 10 -t 1000 testdb : 2642.962465 (including connections
establishing)
pgbench -c 20 -t 1000 testdb : 2253.349999 (including connections
establishing)
pgbench -c 30 -t 1000 testdb : 2037.050831 (including connections
establishing)
pgbench -c 40 -t 1000 testdb : 1918.147928 (including connections
establishing)
pgbench -c 50 -t 1000 testdb : 1773.778845 (including connections
establishing)
pgbench -c 100 -t 1000 testdb : 1153.303103 (including connections
establishing)
pgbench -c 200 -t 1000 testdb : 517.648628 (including connections
establishing)
pgbench -c 300 -t 1000 testdb :  97.442573 (including connections
establishing)

-------------------------------------------------------------------------------
FreeBSD 8.0-STABLE, ZFS, default block size
-------------------------------------------------------------------------------

bonnie++:

Version  1.96       ------Sequential Output------ --Sequential Input-
--Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec
%CP
beastie         24G   120  99 154421  31 81917  16   327  98 196125  16
175.7   4
Latency               222ms   13439ms   12749ms     163ms    1034ms
655ms
Version  1.96       ------Sequential Create------ --------Random
Create--------
beastie             -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec
%CP
                 20 25979  99 32036  55 16342  99 24022  99 +++++ +++ 21384
99
Latency             13639us     107ms     977us   31274us     144us
257us

-------------------------------------------------------------------------------
FreeBSD 8.0-STABLE,ZFS (8k block),postgresql84-server from package,default
conf.
-------------------------------------------------------------------------------

pgbench -c 10 -t 1000 testdb : 2380.858280 (including connections
establishing)
pgbench -c 20 -t 1000 testdb : 2135.241727 (including connections
establishing)
pgbench -c 30 -t 1000 testdb : 2002.773173 (including connections
establishing)
pgbench -c 40 -t 1000 testdb : 1692.844103 (including connections
establishing)

bonnie++:

Version  1.96       ------Sequential Output------ --Sequential Input-
--Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec
%CP
beastie         24G   107  99 119235  22 50495  17   321  98 143257  25
201.5   4
Latency             82593us   13575ms   11274ms   74329us    1398ms
666ms
Version  1.96       ------Sequential Create------ --------Random
Create--------
beastie             -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec
%CP
                 20 24960  99 +++++ +++ 21449  99 22501  99 +++++ +++ 22156
99
Latency             17467us     137us     167us   37528us     110us
147us

-------------------------------------------------------------------------------
FreeBSD 8.0-STABLE,ZFS-atime=off,postgresql84-server from package,default
conf.
-------------------------------------------------------------------------------

bonnie++:

Version  1.96       ------Sequential Output------ --Sequential Input-
--Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec
%CP
beastie         24G   114  99 151906  32 81971  17   295  90 192878  16
173.5   5
Latency               286ms   15766ms   13315ms    1062ms     433ms
727ms
Version  1.96       ------Sequential Create------ --------Random
Create--------
beastie             -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec
%CP
                 20 25295  98 15774  78 12823  99 20979  99 +++++ +++ 22466
99
Latency             30857us   98288us    1321us   35519us     113us
144us

-------------------------------------------------------------------------------
FreeBSD 8.0-STABLE,ZFS-8k/atime=off,postgresql84-server from package,default
conf.
-------------------------------------------------------------------------------

pgbench -c 10 -t 1000 testdb : 2235.009456 (including connections
establishing)
pgbench -c 20 -t 1000 testdb : 1915.160680 (including connections
establishing)
pgbench -c 30 -t 1000 testdb : 1824.546833 (including connections
establishing)
pgbench -c 40 -t 1000 testdb : 1663.443537 (including connections
establishing)

bonnie++:

Version  1.96       ------Sequential Output------ --Sequential Input-
--Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec
%CP
beastie         24G   109  99 123147  24 51160  17   327  97 117123  21
191.4  4
Latency               153ms   17900ms    9873ms     164ms     669ms
600ms
Version  1.96       ------Sequential Create------ --------Random
Create--------
beastie             -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec
%CP
                 20 25324  99 +++++ +++ 18689  99 20837  99 18828 100  5055
100
Latency             13866us   48457us     241us   30971us     199us
596us

-------------------------------------------------------------------------------
FreeBSD 8.0-STABLE,ZFS-8k,postgresql84-server from package,custom config
-------------------------------------------------------------------------------

pgbench -c 10 -t 1000 testdb : 2438.179353 (including connections
establishing)
pgbench -c 20 -t 1000 testdb : 1949.016648 (including connections
establishing)
pgbench -c 30 -t 1000 testdb : 1570.176692 (including connections
establishing)
pgbench -c 40 -t 1000 testdb : 1683.720510 (including connections
establishing)
pgbench -c 50 -t 1000 testdb : 1481.249222 (including connections
establishing)
pgbench -c 100 -t 1000 testdb : 1034.946222 (including connections
establishing)
pgbench -c 200 -t 1000 testdb : 288.125818 (including connections
establishing)
pgbench -c 300 -t 1000 testdb : 57.924377 (including connections
establishing)

-------------------------------------------------------------------------------
FreeBSD 8.0-STABLE,ZFS-8k,postgresql84-server from ports,custom config
-------------------------------------------------------------------------------

pgbench -c 10 -t 1000 testdb : 2252.105155 (including connections
establishing)
pgbench -c 20 -t 1000 testdb : 2065.147771 (including connections
establishing)
pgbench -c 30 -t 1000 testdb : 1762.356143 (including connections
establishing)
pgbench -c 40 -t 1000 testdb : 1832.577548 (including connections
establishing)
pgbench -c 50 -t 1000 testdb : 1571.061549 (including connections
establishing)
pgbench -c 100 -t 1000 testdb : 1091.865609 (including connections
establishing)
pgbench -c 200 -t 1000 testdb : 476.009429 (including connections
establishing)
pgbench -c 300 -t 1000 testdb : 101.918069 (including connections
establishing)

-------------------------------------------------------------------------------
CentOS 5.4, EXT3,postgresql84-server from package, default config
-------------------------------------------------------------------------------

pgbench -c 10 -t 1000 testdb : 2407.934433 (including connections
establishing)
pgbench -c 20 -t 1000 testdb : 2369.103760 (including connections
establishing)
pgbench -c 30 -t 1000 testdb : 2165.642174 (including connections
establishing)
pgbench -c 40 -t 1000 testdb : 2060.112114 (including connections
establishing)

bonnie++:

Version  1.94       ------Sequential Output------ --Sequential Input-
--Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec
%CP
dhcppc0         24G   548  99 167274  30 73785  11  1088  73 217100  11
612.3  17
Latency             15995us     957ms     575ms     576ms     111ms
73868us
Version  1.94       ------Sequential Create------ --------Random
Create--------
dhcppc0             -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec
%CP
                 20 39388  94 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++
+++
Latency               114us     725us     750us     182us      78us
370us

--
View this message in context: http://old.nabble.com/Benchmark-with-FreeBSD-8.0-and-pgbench-tp28569544p28569544.html
Sent from the PostgreSQL - performance mailing list archive at Nabble.com.