Poor disk (virtio) Performance Inside KVM virt-machine vs host machine

 /usr/pgsql-9.4/bin/pg_test_fsync -f test.sync

5 seconds per test

O_DIRECT supported on this platform for open_datasync and open_sync.

Compare file sync methods using one 8kB write:

(in wal_sync_method preference order, except fdatasync

is Linux's default)

        open_datasync                      5190.279 ops/sec     193 usecs/op

        fdatasync                          4022.553 ops/sec     249 usecs/op

        fsync                              3069.069 ops/sec     326 usecs/op

        fsync_writethrough                              n/a

        open_sync                          4892.348 ops/sec     204 usecs/op

Compare file sync methods using two 8kB writes:

(in wal_sync_method preference order, except fdatasync

is Linux's default)

        open_datasync                      2406.577 ops/sec     416 usecs/op

        fdatasync                          4309.413 ops/sec     232 usecs/op

        fsync                              3518.844 ops/sec     284 usecs/op

        fsync_writethrough                              n/a

        open_sync                          1159.604 ops/sec     862 usecs/op

Compare open_sync with different write sizes:

(This is designed to compare the cost of writing 16kB

in different write open_sync sizes.)

         1 * 16kB open_sync write          3700.689 ops/sec     270 usecs/op

         2 *  8kB open_sync writes         2581.405 ops/sec     387 usecs/op

         4 *  4kB open_sync writes         1318.871 ops/sec     758 usecs/op

         8 *  2kB open_sync writes          698.640 ops/sec    1431 usecs/op

        16 *  1kB open_sync writes          262.506 ops/sec    3809 usecs/op

Test if fsync on non-write file descriptor is honored:

(If the times are similar, fsync() can sync data written

on a different descriptor.)

        write, fsync, close                3071.141 ops/sec     326 usecs/op

        write, close, fsync                3303.946 ops/sec     303 usecs/op

Non-Sync'ed 8kB writes:

        write                            251321.188 ops/sec       4 usecs/op

On the host machine:

/usr/pgsql-9.4/bin/pg_test_fsync -f test.sync

5 seconds per test

O_DIRECT supported on this platform for open_datasync and open_sync.

Compare file sync methods using one 8kB write:

(in wal_sync_method preference order, except fdatasync

is Linux's default)

        open_datasync                     11364.136 ops/sec      88 usecs/op

        fdatasync                         12352.160 ops/sec      81 usecs/op

        fsync                              9833.745 ops/sec     102 usecs/op

        fsync_writethrough                              n/a

        open_sync                         14938.531 ops/sec      67 usecs/op

Compare file sync methods using two 8kB writes:

(in wal_sync_method preference order, except fdatasync

is Linux's default)

        open_datasync                      7703.471 ops/sec     130 usecs/op

        fdatasync                         11494.492 ops/sec      87 usecs/op

        fsync                              9029.837 ops/sec     111 usecs/op

        fsync_writethrough                              n/a

        open_sync                          6504.138 ops/sec     154 usecs/op

Compare open_sync with different write sizes:

(This is designed to compare the cost of writing 16kB

in different write open_sync sizes.)

         1 * 16kB open_sync write         14113.912 ops/sec      71 usecs/op

         2 *  8kB open_sync writes         7843.234 ops/sec     127 usecs/op

         4 *  4kB open_sync writes         3995.702 ops/sec     250 usecs/op

         8 *  2kB open_sync writes         1788.979 ops/sec     559 usecs/op

        16 *  1kB open_sync writes          937.177 ops/sec    1067 usecs/op

Test if fsync on non-write file descriptor is honored:

(If the times are similar, fsync() can sync data written

on a different descriptor.)

        write, fsync, close               10144.280 ops/sec      99 usecs/op

        write, close, fsync                8378.558 ops/sec     119 usecs/op

Non-Sync'ed 8kB writes:

        write                            159176.122 ops/sec       6 usecs/op

The file system "inside" and "outside" the same - ext4 on LVM. Disk scheduler  "inside" and "outside" set to "noop". Fstab options same to, setted to rw,noatime,nodiratime,barrier=0. OS on host and guest the same CentOS release 6.5 (Final).

Guest volume options:

Disk bus: Virtio

Cache mode: none

IO mode: native 

Any ideas?