Hi,
> Now Majid stated that he uses "RBD" - Majid, any chance to specify
> what that RBD really is ? What's the tech? What fs? Any ioping or fio
> results? What's the blockdev --report /dev/XXX output ? (you stated
> "high" latency and "high" bandwidth , but it is relative, for me 15ms+
> is high latency and >1000MB/s sequential, but it would help others in
> future if you could specify it by the exact numbers please). Maybe
> it's just a matter of enabling readahead (line in [1]) there and/or
> using a higher WAL segment during initdb.
Unfortunately, I quit that company a month ago (I wish we could
discuss this earlier) and don't have access to the environment
anymore.
I'll try to ask my teammates and see if they can find anything about
the exact values of latency, bw, etc.
Anyway, here is some description of the environment. Sadly, there
are no numbers in this description, but I'll try to provide as much details
as possible.
There is a k8s cluster running over some VMs. Each postgres
instance runs as a pod inside the k8s cluster. So, both the
primary and standby servers are in the same DC, but might be on
different baremetal nodes. There is an overlay network for the pods to
see each other, and there's also another overlay network for the VMs
to see each other.
The storage servers are in the same DC, but we're sure they're on some
racks other than the postgres pods. They run Ceph [1] project and provide
Rados Block Devices (RBD) [2] interface. In order for k8s to use ceph, a
Ceph-CSI [3] controller is running inside the k8s cluster.
BTW, the FS type is ext4.