Hi everyone,
Due to increasing load of my application, I recently changed my
production server from a server with SAS 15k HDDs to a server with intel
SSD disks (only for the postgresql data partition).
It is RAID 10 configuration in both servers.
The rest of the hardware is approximatively the same (maybe a little
more recent on the new server).
I did not changed the postgresql.conf
In a first time, I plugged the new server as a hot standby slave, during
weeks. (I have a master/ slave configuration)
As everything looks good in terms of performances in this slave role,
(latency on /data partition decreased from ~60ms to ~3ms ; and
utilisation of the device decreased significantly too), I decided to
change server roles, and promote as master the server with SSD drives.
(I did several benchmark tests too on both configuration, and SSD config
was always better than old HDD config.)
After the change, I had the following behavior, and I don't understand
why : everything seems to work fine (load is ~6/7, when it was
previously ~25/30 on the HDD server), so the SSD server is faster than
the HDD one, and my apps run faster too, but after some time (can be 5
minutes or 2 hours), the load average increases suddently (can reach 150
!) and does not decrease, so postgres and my application are almost
unusable. (even small requests are in statement timeout)
The only way to decrease load is to restart pooler (pgpool) so that all
requests are stopped, and then the postgres is accessible again, during
some time...
I tried to adapt postgresql.conf parameters to SSD config (random page
cost=1 ), but this does not changed the problem.
When I look IO disk, the RAID volume is 100% busy, which is weird
because if was around 20/30% just before.
Transactions per second is at this moment very low (<10)
I think of a cache issue, but I can't explain why data load should be
longer on SSDs than on HDDs, and I don't know how to find the origin of
the problem.
Does anyone has an idea :) ?
Sorry for my poor english
thanks,
Thomas